Introduction to Search Engines – Part 1

Search engines have changed the way we work with the digital world completely. Digital world around us including the internet has evolved along with search engines almost simultaneously. Here is a primer on the working, technologies, statistics, history, tips for using, searching, etc. for Google.

How do they work?

Search engines work to search across billions of web pages, portals, documents, images, articles, news, journals, etc. to bring relevant and ranked results to your device. They do this by using few concepts like META Tags, Linking of web pages & content with each other, citations, source, web page credibility, social media relationships, configurations on websites like robots.txt / sitemap, etc.

Statistics

What you see as result on the first page of Google India is not the same in Google Europe for the same search word. Search results are customized and localized for each country, individual, session and even device at times. Currently as per publicly available statistics Google gets 40,000+ search requests per second which translates to few billion searches per day and would touch almost 1 Trillion searches approximately per year.

History of Google

Derived from word Googol and started at Stanford University. Initial algorithm was named BackRub.

Technology

Search engines have a spider / crawler which crawls across the internet to search and index / store the data in various databases. Ranking algorithm ranks all the content and refines it as per various parameters. Clustered servers across the world process the requests of search and provide content to devices. Lot of optimization based on new searches and corrections for old search terms / updating based on content changes happen regularly.

Tips & Tricks

Try: <vs.> <“EXACT SEARCH TERM”> <“SEARCH TERM -WITH SPECIFIC WORD EXCULED”> <“SITE: Search within a site”> <“DEFINE:”> <“SearchTerm1 * SearchTerm2 – Search everything including both terms and combinations with anything inbetween”>

Explore options of SETTINGS & TOOLS especially in IMAGE Search

Google Scholar

Search for scholarly papers, articles, theses, journals, books, reports, patents, etc.

Alternative search engines

Bing, DuckDuckGo, Yippy, Ask, DogPile, Yandex, etc.

If you want to learn more about the technologies used by search engines, try out:

Apache Solr

Apache Nutch

In Part – 2 we will discuss advanced topics like AdWords, AdSense, Google Analytics, Google Search Console, new implications of rankings based on https / SSL, details of Nutch & Solr, algorithms for search, other search engines like Yahoo, Bing, etc.

Facebook Privacy & Security tips

Many of us use Facebook for various reasons like socializing on the internet, managing and participating in groups, creating and managing company pages, keeping in touch with friends and colleagues, sharing our updates & pictures, etc. But many don’t pay attention to privacy and security aspects of using Facebook.

Here are some pointers on what areas of privacy & security you should be looking at in Facebook. This post does not give you what setting / configuration you should be keeping / selecting rather it makes you aware about those settings and you should select / configure the option based on your requirements.

  • Explore OTP / Two factor authentication using mobile / smartphone number / authentication app
  • Explore notifications over email & mobile
  • Explore options for your Cover & Profile picture – there is option to show your Profile picture to only your friends or network
  • Check whether your default posts are Public / shown to friends / shown to friends of friends / shown to friends except acquaintances
  • Check the type of groups you join: Public / Closed / Secret
  • Explore the privacy options like LIMIT YOUR OLD POSTS, Who can search you by email / phone, Who can see your friends list, etc.
  • Explore the options for people to Tag you in their posts
  • Explore the option to create custom username
  • Check the Location History option
  • Check which profiles in your network are Inactive / Spam / Without data / Suspect / Unknown. It’s best to remove them from your network by clicking UNFRIEND. Stay in touch with them over other mediums like Skype, LinkedIn, WhatsApp, SMS, Email, Signal, Google Hangout, etc.
  • Check the Apps & Website integration under Settings
  • Check the locations from where you are logged in under settings
  • Check your public posts by logging out & checking your profile or using the VIEW AS option under your profile
  • Check the setting whether search engines should link to your profile under privacy
  • Check the information in your profile added under ABOUT and who it is visible to on all sections and details of profile – Friends / Friends of Friends / Public / etc.
  • Check manage sections option in your profile and check privacy options for each of those sections
  • Check your advertisement preferences
  • Check the options which show your relationship with others under profile – whether it’s only shown to you, friends or friends of friends or public
  • Set up Trusted Contacts to recover your account
  • For your Facebook Mobile App check what permissions it has been given by the operating system (Android or iOS) under App permissions and review them based on your need
  • Check the people who are added as ADMIN for your company page if you manage one
  • There are a host of other options in settings for company pages too from which you can manage options like – from which country the visitors can see the page, whether tagging is allowed, age profile of your visitors, etc. If you are managing a company page you should explore them too

Security tips for Google Drive & Android

Google Drive:

  • Open Google Drive on web based browser like Chrome
  • On each of the folders if you see a person symbol it’s shared, if you don’t see the symbol its not shared
  • For each folder -> click SHARE -> Advanced -> Check which user has view rights and which user has edit rights -> Disable options to download, print, and copy for commenters and viewers -> Prevent editors from changing access and adding new people.
  • For each folder -> click SHARE -> Advanced -> SHARE SETTINGS -> Change -> Check settings there are 4 to 5 options
  • Note -> Even if your folder is not shared, files inside it can be in shared mode – This seems to be a big missing feature in Google Drive. There is no clear way to check this in one go for all files and we individually need to check each file. Workaround: Write Java code using Google Developers API to check this: https://developers.google.com/drive/

Android security settings to explore:

  • Emergency info – Settings -> Users -> Emergency info -> INFO & CONTACT – Note: Can be seen on locked screen
  • Settings -> Security -> Screen Lock
  • Encryption ON – at-least Android 6 required, by default it’s on from 6/7 onward – Settings -> Security
  • Settings -> Security -> Screen Lock -> Gear button -> Lock message
  • Security -> Install from unknown locations -> Should be OFF
  • Settings -> Users -> Guest user -> Turn on phone calls -> OFF
  • Settings -> Users -> Add users while screen is locked -> OFF
  • Settings -> Security -> Make password visible -> OFF
  • Settings -> Security -> Set up SIM card lock

Do educational institutes fulfill their obligation to supply proper workforce?

This question is negatively answered for most of the institutes across the globe. India is not an exception.

There is wide gap in the skills required by the industry vis-à-vis supplied by educational institutes. Over 75 per cent graduates are not employable.

What are the reasons for this gap, quantum and the nature of gap? It is observed that there is no link between the institutes and industry. Institutes are lagging to industry. Industry is actively working to beat the competition and hence brings innovations in the products, processes, strategies, business models and techniques.

Industry creates knowledge through research; it uses research in real life to solve problems. Industry pays for solving problems, this
motivates employee to perform. It is a question of survival and growth for industry and employee.

Institutes are designing curricula based on what has happened in the industry, rather than what will happen in the industry or market. Institutes have very narrow focus while they teach students. They are governed by rules rather than reality. The curriculum focuses on prescribed and sink knowledge.

Industry requires dynamism. Institutes teach to solve existing problems. Industry requires people who can solve new problems. Teacher and students’ performance is measured on wrong calibration of examinations.

Examination systems are built to test memories rather than ability to solve real life problems. Subject knowledge is imparted in isolation.

A good student may be capable to solve a problem in isolation, but real life situation in industry is complex; and hardly in isolation.  Real life problems are complex, dynamic and resolved through team work. Students are prepared and tested in isolation.

Team work and communication in industry is typical and sine-qua-non for success in industry. Education institutes fail to realize and gauge this problem.

How to build a lean startup in a Tier 2 or 3 city in India ? – Part 1

If you are exploring an option to start an IT / Digital startup in a Tier 2 or 3 city in India and are searching for some advice on how to do that in a lean, cost controlled way – here is some help:

  1. Register your company preferably in one of the three modes: Limited Liability Partnership, One Person Company OR Private Limited. There is a 4th option of proprietorship as well which is the easiest with less compliance requirements but with some other disadvantages. Select the one which suits your revenue, risks, appetite, customers and few other factors like in which line of business you are. You will need help of a CA and lawyer. Should include GST registration, Udyog Aadhaar, etc. Typical cost – Up to Rs. 20,000 plus invested amount for paid up capital – one time cost.
  2. You will need a business plan to chart out your path for at-least 1 to 2 years in the future. What will you offer: Products / Services / Or a mix of both? Where will you get the revenue from? Where will the funding come from? Who all will be your customers? What is the geography you will serve? Who will be your partners in management team? What partnerships in software / hardware with various companies do you need to have? How will you market & sell? What are the things you will outsource vs. do in-house? Where will the growth come from? Where are the future opportunities in the business space you want to serve? These are some basic questions that need to be answered in the business plan.
  3. Build your website preferably by investing time and taking help of another IT startup or SME. If you have the skills you can build it on your own. Typical cost for hosting & website, emails, some tools, cloud applications like project / requirement / salary management, storage, collaboration, CRM, etc. – up to Rs. 50,000 to 70,000 per year depending on what options, products, etc. you select.
  4. Register your Trade Mark. Typical cost – Rs. 10,000 to Rs. 50,000 depending on what all classes you file in and what are the further steps / objections. Once in 10 years.
  5. You will incur expense on office like properly lease, electricity & salaries along with maintenance of current account, debit cards, internet banking, etc. – Depends on the city and location along with employee strength – Typically Rs. 75,000 to 200,000 per month for 3 to 5 employee company in Tier 2 & 3 cities.
  6. You will need to meet legal & compliance requirements for filing taxes, annual meetings, audit, lawyer, etc. via a Chartered Accountant – Rs. 40,000 per year.
  7. Digital marketing & sales activities will be required to be done regularly. Here are some major ones: Advertisements on Google AdWords, Promotions on Facebook, Twitter, LinkedIn, etc. – Typically Rs. 50,000 to 75,000 per year.
  8. You will need to maintain Internet, computers and other associated administrative activities. Expenses – Rs. 20,000 per month.
  9. Setup and regular filing / maintenance for EPF / EPS, Employee medical / life / accident / professional liability insurance will need to be done. Related expenses – Depends on the size of your firm and revenue. But a reasonable amount for a small company of 10 employees is about Rs. 15,000 to 25,000 per month. Some of these are optional depending on the size of your company.

My suggestion is to go with a Tier 2 or 3 city as your base and use digital medium to offer your services along with usage of online products on taxation, legal, compliance, audit, salaries, collaboration, sales, marketing, etc. to manage your company as much as possible with as small a footprint as possible in physical world to start your venture.

You can save a lot on office lease, physical sales & meetings, fuel, meeting customers, collaboration and managing data if you go digital.

Reach out to me for any discussions, help, suggestions on building a lean and low cost startup. Good luck ! !

Skills required for working as a Solution Architect

  • High level Architecture understanding about Clustering of Databases / Applications Servers / Operating Systems, Enterprise Architecture, Non Functional Requirements, Integrations, Various Enterprise Software Products & Frameworks
  • Concepts of Virtualization and Cloud platforms like AWS, Azure & Google Cloud
  • Integration – Tools, Frameworks & Products that can be used like ESB, Web-Services, etc.
  • Latest trends around Big Data, NoSQL, React, Mobility, Portals, ERP, Cloud, Search, Scanning, Analytics, Networking, etc.
  • Reading list of websites for updating oneself on various domains, trends & technologies
  • Documents, spreadsheets & presentations – Skills with MS-Word, MS-Excel, MS-PPT, Google Docs, HTML, Visio, Free / Open Source / Cloud based Enterprise Architecture & Digital Marketing Tools & MS Paint, etc.
  • Programming skills in at-least one front line full application programming language like Java, VB.Net, C#, etc. as a minimum & concepts of digital electronics & microprocessors
  • Database, Encryption & Computer Networking skills including SQL, PL/SQL, Indexing, Triggers, Procedures, TCP/IP, UDP, IPv6/4, SSL/HTTPS, Types of encryption, etc.
  • Basics of how to create requirement documents, proposals, estimations & statement of works
  • Soft skills for working & collaborating with various stakeholders from Top Management, HR, Finance, Legal, Operations to developer level and customers
  • Reporting skills around creation of document, status updates, etc.
  • Project Management & Testing overview, Tools, Processes & Products used for the same
  • Overview of Quality Frameworks like CMMi, ISO, etc.
  • Ability to learn things quickly, present them in concise, customer friendly ways & networking skills to support Sales & Project Managers
  • Basic understanding of RFP / RFI / RFQ & Consulting process

Part 1 for this here: What do Presales Consultants / Solution Architects do ?

General Data Protection Regulation (GDPR) – Layman’s Introduction

  • Data of any EU citizen / entity cannot be shared without explicit consent. Terms & Conditions / Privacy checkbox doesn’t suitably give consent as per this law. Explicit consent is required
  • Records of when consent was given, how, what time, when revoked, etc. must be stored
  • Any breach of data or hacking must be informed to the stakeholders within 72 hours
  • It applies to whole of EU
  • Consent can be withdrawn anytime & when they want, and data processor / controller must comply
  • There is an option for right to be forgotten. If someone says to a data processor / controller that “FORGET ME” they must remove all data on them. It applies to all places which store & process / control the relevant data
  • GDPR applies to Production environment of Digital Systems which have data on EU citizens or entities. If Test / Pre-Production environment has unmasked data of EU citizens / entities again this law applies
  • https://www.zdnet.com/article/gdpr-an-executive-guide-to-what-you-need-to-know/ 

What should managers be doing in IT industry – Part 1

There are many things that senior engineers and managers do well in IT industry among which few are listed below:

  • Architecture
  • Presales
  • Solutioning
  • Debugging and solving problems
  • Exploring new technologies around the areas they have learned in the past
  • Analysis of scenarios
  • Product Management
  • Project Management
  • Setting up processes
  • Team Management
  • Digital Marketing & Sales

Some things that they don’t do well:

  • Put in long hours in support & maintenance
  • Explore completely new areas in which they have not worked before
  • Testing
  • Programming
  • Learning new trends and technologies

If you are born in 1980s or before, you would have become a manager in IT industry by now. In this case, at times many stop updating themselves.

Below are some pointers on how to update oneself and rebuild your career:

  • Instead of only reading about domains and technologies, experiment with software practically
  • Setup a small consulting / teaching / area specific business where you have passion, create a website, host some parts on traditional shared servers and some on AWS / Azure / etc.
  • Do some freelancing on the sides
  • Meet startups, founders, discuss their problems and offer free consulting hours to solve their issues
  • Network with local young IT professionals and see the latest apps and trends
  • Attend various MeetUps in your area
  • Pick up a new programming language and product
  • Try out Linux and other open source projects from SourceForge, GitHub, Apache Foundation, etc.
  • Take up courses from Coursera, edX, Udemy and similar portals
  • Go for further education in part time, online or distance mode – courses like MBA, MCA, Executive MBA, MSc, MS online from Universities across the world, MTech, PhD, etc. which can be done with your job
  • Follow industry professionals in upcoming technologies and domains on LinkedIn, Twitter and Facebook or other social media platforms to get updates

Unless one keeps up with the times they risk becoming obsolete !!

Advantages & Disadvantages of Crypto-Currencies

List of some of the major Crypto-Currencies as of now:

  • Bitcoin
  • Ripple
  • Ethereum
  • Litecoin
  • Zcash
  • Dash
  • Monero
  • More are on the way

Advantages of Crypto-Currencies:

  • Decentralized
  • Digital
  • Easy access
  • Traded on exchanges
  • Block-chain or similar modern database & protocol based non modifiable ledger of transactions
  • Immediate settlement and difficult to reverse / refuse payment
  • Lower fees
  • Difficult to do identity theft for conducting fraudulent transactions

Disadvantages of Crypto-Currencies:

  • Less or no regulation
  • No or less audits of exchanges where they trade
  • Price discovery does not follow standard financial rules
  • Mining is ad-hoc and affects money supply
  • Difficulty in reversal of payments in case of mistakes
  • Price volatility
  • Less clarity on taxation in some countries
  • Anonymity
  • Cutting edge technology which is yet to stabilize
  • Functionally features are complex for layman to understand

Information security tips while working with digital sources and internet

  • Setup OTP and recovery emails for all your accounts
  • Check permissions that various apps have on your phone
  • When you get an option – TRUST THIS DEVICE in Gmail or similar accounts, only select this option on your personal devices not public devices
  • Keep your desktop, laptop & mobiles password protected & encrypted if possible
  • Use standard Anti-Virus like McAfee, Windows Defender, Symantec, Norton, etc. Note: Android phones also have anti-virus
  • Explore and use a good VPN service
  • Go to your ACCOUNT settings regularly in Linkedin, Facebook, Gmail, etc. and see the logged in sessions (Who is using your account) and check which apps are integrated with your account, what information they are taking out – typically they take your friend’s list / relatives, age, email, phone number, etc.
  • Learn to backup your data in Cloud or external Hard Disk or both
  • Setup locate your phone, remote wipe, virus scan, link check, capture pic on wrong password attempts, etc. especially on Android via Anti-Virus or other means.
  • Update your devices across laptop, desktop and mobiles regularly using the official update process
  • Do not root your mobile devices
  • In Android don’t switch on the feature to TRUST UNKNOWN APPS to install APKs. Always install APPs from Android Play Store and Apple App Store only
  • Keep Bluetooth, WIFI, NFC off when not in use
  • When you leave your home switch off your WIFI
  • Change your passwords of WIFI, emails, accounts once in 6 months at-least. Your WIFI at home via router should be on encrypted network not non-encrypted, check the ADMIN page. Change the ADMIN password of your router at home, mostly many don’t change the username/password from admin/admin which is an easy entry point for wrong use. Update your router software also regularly via the official update option
  • For storing your passwords either use a diary / notebook OR there are encrypted software like password managers / wallets which store your passwords safely as an APP in your mobile – use them. See their ratings in PLAY STORE / APP STORE and then install or use. Don’t store passwords in plain text on computer or mobile.
  • Don’t keep same passwords across all your accounts – if you do so and one gets compromised all others get compromised

Ideas on Innovation around Technology. We Thrive On Ideas. We are Learner Centered, Open Source & Digital Focused.