History & Future of Information Technology

Late 1970s – Ethernet

Late 1980s & Early 1990s – Internet

Late 1990s – Y2K was the trend and revolution of easy operating systems like Windows with word processing and spreadsheets happened. Also, LINUX. ERP, CRM, BPM, HRM and similar software pick up in market.

Early 2000s – Internet boom, Email, Browsers, Dot Com, Start of E-Commerce, Java, boom in Portals, Scanning solutions, Enterprise Content Management, Search and associated technologies

Post 2011- Shift to Mobiles, GPS, Location tracking, cameras, virtual / augmented reality on mobiles and so on. Lot of growth for companies like Google / Apple.

Post Mobiles – Cloud, Machine Learning, Artificial Intelligence, Analytics, IoT, Beacons, Business Intelligence & BigData

2010/11 onwards – RPA, SAAS/PAAS/IAAS, Facebook, Social Media, Agile, DevOps, Crypto currency, Focus on Internet, Network & Cloud security

Time is now right to have an AI / ML based System in Enterprise IT ecosystem that can create and manage systems on it’s own or at least most of it at Enterprise / Technical & Solution Architecture level along with automated creation, deployment and maintenance of software systems using Cloud

Reach out to me at neil@techandtrain.com if you want to discuss Data Science / R / Java / etc. or want to conduct a training for MBA / BE / MCA / MSc students or are interested in having a workshop for your managers / executives on Data Science / R / Java / AWS / Excel / etc.

Learning R Programming – Part 1

As per Glassdoor Top 5 skills in Data Science for job openings are:

  1. Python
  2. R
  3. SQL
  4. Hadoop
  5. Java

Most Java developers know SQL, Hadoop & Java to a good extent in today’s environment, two important skills Python & R should be learned by the Java developer / architect / manager if s/he wants to contribute / work in Data Science area. In this article you will find a structured step by step approach for learning programming in R.

  1. To start with install R and familiarize yourself with R Console & R Script interface. You can run commands on both but it’s best to write multiple commands and try them out in script editor. Use short cut CTRL + R to run your commands in R Script editor.
  2. Explore menu options like Package -> Install / Load / Choose CRAN Mirror. By default many commands for statistics, visualization, etc. are given in R by default. Big set of libraries are already loaded into R by default and 100s more are available. Select any mirror to download new packages and install / load them step by step. You will need working internet connection. Learn how to set working directory. You can see default libraries available in R using library()
  3. From there move on to various R Objects / Data Types – Explore Data Frame, Vector, List, Matrices, Arrays and Factors. Try out examples for the same.
  4. Next step learn to load / read and write datasets by commands like read.csv / write.csv. You can also read / write excel sheets but for it you will need other packages. See the basic commands like summary, structure & fix to analyze / edit your dataset
  5. Next step – go through various categories of operators (logical, mathematical, relational) and concepts like pipe %>%, constants, rules for naming identifiers followed by various statistical functions directly available in R. You can get help on a command by using ?<COMMAND>. Also, learn to create functions and use conditions like if
  6. By now you should revise basics of statistics & various visualization charts which are taught typically in Year 1 / Semester 1 of MBA. Explore various default commands for statistics built into R by default. Some examples – mean, variance, standard deviation, etc.
  7. Learn to manipulate / read / write datasets using subset, sample_n & sample_frac and using dplyr package which has commands like select & filter among others
  8. Check various types of default visualization commands in R for various charts like barplot & pie. Post this learn how to use ggplot2 package
  9. You will get many datasets at kaggle.com and various websites like stock exchanges – NSE / BSE, RBI, Open Data websites of various Governments and others
  10. Explore top 20 packages of R categorized by various areas as given below.

An advantage of learning R is that you will become better at statistics & data science. It’s much simpler than Java in terms of syntax and structure and is influenced by open source languages / scripting like Linux, etc.

Reach out to me at neil@techandtrain.com if you want to discuss R, conduct a training for MBA / BE / MCA / MSc students in R or want to conduct a workshop for your managers / executives on Data Science / R / Java / etc.

References:

Top 10 skills for Data Science – Glassdoor Economic Research

Top 20 packages in R

Universities & Colleges offering courses & degrees in Analytics / Data Science in India – Part 1

  • BITS Pilani – Offering MSc in Analytics, MTech in Computer Science with specialization in Analytics
  • IIM Calcutta, ISI Kolkata & IIT Kharagpur – PG Diploma in Business Analytics
  • Great Lakes, Chennai – PGP in Analytics
  • IIIT Bangalore – PGP in Analytics via UpGrad
  • XLRI – Executive Program in Data Science
  • NIRMA University – MBA – Courses in 2nd year MBA covering Analytics & Data Science
  • Shanti Business School, Ahmedabad – PGDM having 2nd year specialization in Analytics / Decision Science
  • Gujarat University – MSc in Machine Learning & Artificial Intelligence
  • PG Diploma in Data Science – Manipal Global Academy in Data Science
  • Big Data & Visual Analytics – SP Jain School of Global Management
  • Master of Business Administration (Data Sciences and Data Analytics)- Symbiosis Center for Information Technology
  • Indian Institute of Information Technology and Management – Kerala (IITM-K) M.Sc. In Computer Science with Specialization in Data Analytics
  • Indian Institute of Science, Bangalore M.Tech. In Computational and Data Science & M.Tech.(Research) / PhD
  • Executive Program in Business Analytics- IIM Calcutta
  • St. Xavier’s College (Autonomous), Ahmedabad. (Collaboration with Tata Consultancy Services Limited) – M.Sc. in Big Data Analytics
  • IIIT Hyderabad – MTech in Data Science / Analytics

Not a complete list but gives you good data points on courses / degrees on Data Science / Analytics in India

Mystery of ‘Perfect Numbers’ Resolved – Perfect Number is Always Even and Predictable

Mystery of ‘Perfect Numbers’ Resolved – Perfect Number is Always Even and Predictable

Vrajlal Sapovadia (Ph.D.)
Sweta Patel (Ph.D.)

Introduction

In number theory, a perfect number is a positive integer that is equal to the sum of its proper positive divisors, excluding the number itself. In other words, a perfect number is a number that is half the sum of all of its positive divisors (including itself) i.e. σ1(n) = 2n. To explain in practical terms, we elaborate first few Perfect Numbers. It may be noted that ‘Perfect Numbers’ are sparse are thinly dispersed. Starting from 3rd Century BC, mathematicians are working on Perfect Numbers. Till April 2018, i.e. during last 2300 years active research, researchers could recognize only 50 perfect numbers. There are 2 perfect numbers in first 100 and 4 in first million. Absolute distance between two perfect numbers increase exponentially as you go higher to the next perfect number[1]. One can find at least one perfect number till 4 digit numbers, and then it becomes a real rarity. Subsequent perfect numbers appears at 8, 10, 12 and 19 digits. 15th perfect number has 770 digits while 16th have 1327 digits. 25th perfect number has 13066 digits. 50th perfect number has 46,498,850 digits.

The current literature is still debating on two issues:

  1. Can perfect number is predictable?
  2. Can perfect number be odd?

We argue that perfect number is predictable and we have developed a formula which answers both lead questions as follow:

  1. Perfect number is predictable
  2. Perfect number is always even

Predictability

Euclid proved that 2p−1(2p − 1) is an even perfect number whenever 2p − 1 is prime (Euclid, Prop. IX.36). the first four perfect numbers are generated by the formula 2p−1(2p − 1), with p a prime number, as follows:

for p = 2:   21(22 − 1) = 6

for p = 3:   22(23 − 1) = 28

for p = 5:   24(25 − 1) = 496

for p = 7:   26(27 − 1) = 8128.

Prime numbers of the form 2p − 1 are known as Mersenne primes, after the seventeenth-century monk Marin Mersenne, who studied number theory and perfect numbers. For 2p − 1 to be prime, it is necessary that p itself be prime. However, not all numbers of the form 2p − 1 with a prime p are prime; for example, 211 − 1 = 2047 = 23 × 89 is not a prime number.[11] In fact, Mersenne primes are very rare—of the 2,270,720 prime numbers p up to 37,156,667,[12] 2p − 1 is prime for only 45 of them.

Nicomachus (60–120 AD) conjectured that every perfect number is of the form 2p−1(2p − 1) where 2p − 1 is prime.[13] Ibn al-Haytham (Alhazen) circa 1000 AD conjectured that every even perfect number is of that form.[14] It was not until the 18th century that Leonhard Euler proved that the formula 2p−1(2p − 1) will yield all the even perfect numbers. Thus, there is a one-to-one correspondence between even perfect numbers and Mersenne primes; each Mersenne prime generates one even perfect number, and vice versa. This result is often referred to as the Euclid–Euler theorem. As of January 2018, 50 Mersenne primes are known,[15] and therefore 50 even perfect numbers (the largest of which is 277232916 × (277232917 − 1) with 46,498,850 digits).

Owing to their form, 2p−1(2p − 1), every even perfect number is represented in binary as p ones followed by p − 1  zeros. Interestingly, when a perfect number is converted into binary, it is not only a pernicious number, but binary sequence is spectacular having all 1 on the left side followed by all 0. Interestingly count of 1 is a prime number (p) and 0 is p-1.

610 = 1102

1 (p = 2) and 0 (1)

2810 = 111002

1 (p = 3) 0 (2)

49610 = 1111100002

1 (p = 5) 0 (4)

812810 = 11111110000002

1 (p = 7) 0 (6)

3355033610 = 11111111111110000000000002

1 (p = 13) 0 (12)

858986905610 = 1111111111111111100000000000000002

1 (p = 17) 0 (16)

13743869132810 = 11111111111111111110000000000000000002

1 (p = 19) 0 (18)

230584300813995212810 = 11111111111111111111111111111110000000000000000000000000000002

1 (p = 31) 0 (30)

Thus every even perfect number is a pernicious number. Note that every even perfect number is also a practical number. Therefore a formula to find a perfect number can be developed as 1….(p) 0…(p-1), where 1 (p) and 0 (p-1) are binary symbol. Thus, a binary number so written equal to a PRIME (p) ‘1’ followed by p-1 ‘0’ could be a perfect number. It may be noted that all prime count does not result into perfect number. Therefore, it is pertinent to test each prime number with formula[2] will establish whether resultant number is perfect number or not. But in any case, this will reduce substantially the experiment time to find next perfect number or this formula provides a lead to find perfect number with less experiment time.

Odd vs. Even

It is unknown whether there is any odd perfect number, though various results have been obtained. In 1496, Jacques Lefèvre stated that Euclid’s rule gives all perfect numbers, thus implying that no odd perfect number exists. More recently, Carl Pomerance has presented a heuristic argument suggesting that indeed no odd perfect number should exist. All perfect numbers are also Ore’s harmonic numbers, and it has been conjectured as well that there are no odd Ore’s harmonic numbers other than 1. An exhaustive search by the GIMPS[3] distributed computing project has shown that the first 46 are all even numbers represented by 2p−1(2p − 1).

First 50 perfect numbers listed[4] are all even and their last one or two digits are always 6 or 28. In support of arguments made by various researchers, we found that a perfect number can be presented as binary with formula 1….(p) 0…(p-1). Any binary pattern as 1 (n) 0 (n-1) will always result into even number. Therefore any perfect number is always a even number.

[1] 6 (1), 28 (2), 496 (3), 8128 (4), 33550336 (8), 8589869056 (10), 137438691328 (12), 2305843008139952128 (19)
[2] Binary numbers 1….(p) 0…(p-1)
[3] The Great Internet Mersenne Prime Search (GIMPS) is a collaborative project of volunteers who use freely available software to search for Mersenne prime numbers.
[4] https://en.wikipedia.org/wiki/List_of_perfect_numbers

Introduction to Search Engines – Part 1

Search engines have changed the way we work with the digital world completely. Digital world around us including the internet has evolved along with search engines almost simultaneously. Here is a primer on the working, technologies, statistics, history, tips for using, searching, etc. for Google.

How do they work?

Search engines work to search across billions of web pages, portals, documents, images, articles, news, journals, etc. to bring relevant and ranked results to your device. They do this by using few concepts like META Tags, Linking of web pages & content with each other, citations, source, web page credibility, social media relationships, configurations on websites like robots.txt / sitemap, etc.

Statistics

What you see as result on the first page of Google India is not the same in Google Europe for the same search word. Search results are customized and localized for each country, individual, session and even device at times. Currently as per publicly available statistics Google gets 40,000+ search requests per second which translates to few billion searches per day and would touch almost 1 Trillion searches approximately per year.

History of Google

Derived from word Googol and started at Stanford University. Initial algorithm was named BackRub.

Technology

Search engines have a spider / crawler which crawls across the internet to search and index / store the data in various databases. Ranking algorithm ranks all the content and refines it as per various parameters. Clustered servers across the world process the requests of search and provide content to devices. Lot of optimization based on new searches and corrections for old search terms / updating based on content changes happen regularly.

Tips & Tricks

Try: <vs.> <“EXACT SEARCH TERM”> <“SEARCH TERM -WITH SPECIFIC WORD EXCULED”> <“SITE: Search within a site”> <“DEFINE:”> <“SearchTerm1 * SearchTerm2 – Search everything including both terms and combinations with anything inbetween”>

Explore options of SETTINGS & TOOLS especially in IMAGE Search

Google Scholar

Search for scholarly papers, articles, theses, journals, books, reports, patents, etc.

Alternative search engines

Bing, DuckDuckGo, Yippy, Ask, DogPile, Yandex, etc.

If you want to learn more about the technologies used by search engines, try out:

Apache Solr

Apache Nutch

In Part – 2 we will discuss advanced topics like AdWords, AdSense, Google Analytics, Google Search Console, new implications of rankings based on https / SSL, details of Nutch & Solr, algorithms for search, other search engines like Yahoo, Bing, etc.

Facebook Privacy & Security tips

Many of us use Facebook for various reasons like socializing on the internet, managing and participating in groups, creating and managing company pages, keeping in touch with friends and colleagues, sharing our updates & pictures, etc. But many don’t pay attention to privacy and security aspects of using Facebook.

Here are some pointers on what areas of privacy & security you should be looking at in Facebook. This post does not give you what setting / configuration you should be keeping / selecting rather it makes you aware about those settings and you should select / configure the option based on your requirements.

  • Explore OTP / Two factor authentication using mobile / smartphone number / authentication app
  • Explore notifications over email & mobile
  • Explore options for your Cover & Profile picture – there is option to show your Profile picture to only your friends or network
  • Check whether your default posts are Public / shown to friends / shown to friends of friends / shown to friends except acquaintances
  • Check the type of groups you join: Public / Closed / Secret
  • Explore the privacy options like LIMIT YOUR OLD POSTS, Who can search you by email / phone, Who can see your friends list, etc.
  • Explore the options for people to Tag you in their posts
  • Explore the option to create custom username
  • Check the Location History option
  • Check which profiles in your network are Inactive / Spam / Without data / Suspect / Unknown. It’s best to remove them from your network by clicking UNFRIEND. Stay in touch with them over other mediums like Skype, LinkedIn, WhatsApp, SMS, Email, Signal, Google Hangout, etc.
  • Check the Apps & Website integration under Settings
  • Check the locations from where you are logged in under settings
  • Check your public posts by logging out & checking your profile or using the VIEW AS option under your profile
  • Check the setting whether search engines should link to your profile under privacy
  • Check the information in your profile added under ABOUT and who it is visible to on all sections and details of profile – Friends / Friends of Friends / Public / etc.
  • Check manage sections option in your profile and check privacy options for each of those sections
  • Check your advertisement preferences
  • Check the options which show your relationship with others under profile – whether it’s only shown to you, friends or friends of friends or public
  • Set up Trusted Contacts to recover your account
  • For your Facebook Mobile App check what permissions it has been given by the operating system (Android or iOS) under App permissions and review them based on your need
  • Check the people who are added as ADMIN for your company page if you manage one
  • There are a host of other options in settings for company pages too from which you can manage options like – from which country the visitors can see the page, whether tagging is allowed, age profile of your visitors, etc. If you are managing a company page you should explore them too

Security tips for Google Drive & Android

Google Drive:

  • Open Google Drive on web based browser like Chrome
  • On each of the folders if you see a person symbol it’s shared, if you don’t see the symbol its not shared
  • For each folder -> click SHARE -> Advanced -> Check which user has view rights and which user has edit rights -> Disable options to download, print, and copy for commenters and viewers -> Prevent editors from changing access and adding new people.
  • For each folder -> click SHARE -> Advanced -> SHARE SETTINGS -> Change -> Check settings there are 4 to 5 options
  • Note -> Even if your folder is not shared, files inside it can be in shared mode – This seems to be a big missing feature in Google Drive. There is no clear way to check this in one go for all files and we individually need to check each file. Workaround: Write Java code using Google Developers API to check this: https://developers.google.com/drive/

Android security settings to explore:

  • Emergency info – Settings -> Users -> Emergency info -> INFO & CONTACT – Note: Can be seen on locked screen
  • Settings -> Security -> Screen Lock
  • Encryption ON – at-least Android 6 required, by default it’s on from 6/7 onward – Settings -> Security
  • Settings -> Security -> Screen Lock -> Gear button -> Lock message
  • Security -> Install from unknown locations -> Should be OFF
  • Settings -> Users -> Guest user -> Turn on phone calls -> OFF
  • Settings -> Users -> Add users while screen is locked -> OFF
  • Settings -> Security -> Make password visible -> OFF
  • Settings -> Security -> Set up SIM card lock

Do educational institutes fulfill their obligation to supply proper workforce?

This question is negatively answered for most of the institutes across the globe. India is not an exception.

There is wide gap in the skills required by the industry vis-à-vis supplied by educational institutes. Over 75 per cent graduates are not employable.

What are the reasons for this gap, quantum and the nature of gap? It is observed that there is no link between the institutes and industry. Institutes are lagging to industry. Industry is actively working to beat the competition and hence brings innovations in the products, processes, strategies, business models and techniques.

Industry creates knowledge through research; it uses research in real life to solve problems. Industry pays for solving problems, this
motivates employee to perform. It is a question of survival and growth for industry and employee.

Institutes are designing curricula based on what has happened in the industry, rather than what will happen in the industry or market. Institutes have very narrow focus while they teach students. They are governed by rules rather than reality. The curriculum focuses on prescribed and sink knowledge.

Industry requires dynamism. Institutes teach to solve existing problems. Industry requires people who can solve new problems. Teacher and students’ performance is measured on wrong calibration of examinations.

Examination systems are built to test memories rather than ability to solve real life problems. Subject knowledge is imparted in isolation.

A good student may be capable to solve a problem in isolation, but real life situation in industry is complex; and hardly in isolation.  Real life problems are complex, dynamic and resolved through team work. Students are prepared and tested in isolation.

Team work and communication in industry is typical and sine-qua-non for success in industry. Education institutes fail to realize and gauge this problem.

How to build a lean startup in a Tier 2 or 3 city in India ? – Part 1

If you are exploring an option to start an IT / Digital startup in a Tier 2 or 3 city in India and are searching for some advice on how to do that in a lean, cost controlled way – here is some help:

  1. Register your company preferably in one of the three modes: Limited Liability Partnership, One Person Company OR Private Limited. There is a 4th option of proprietorship as well which is the easiest with less compliance requirements but with some other disadvantages. Select the one which suits your revenue, risks, appetite, customers and few other factors like in which line of business you are. You will need help of a CA and lawyer. Should include GST registration, Udyog Aadhaar, etc. Typical cost – Up to Rs. 20,000 plus invested amount for paid up capital – one time cost.
  2. You will need a business plan to chart out your path for at-least 1 to 2 years in the future. What will you offer: Products / Services / Or a mix of both? Where will you get the revenue from? Where will the funding come from? Who all will be your customers? What is the geography you will serve? Who will be your partners in management team? What partnerships in software / hardware with various companies do you need to have? How will you market & sell? What are the things you will outsource vs. do in-house? Where will the growth come from? Where are the future opportunities in the business space you want to serve? These are some basic questions that need to be answered in the business plan.
  3. Build your website preferably by investing time and taking help of another IT startup or SME. If you have the skills you can build it on your own. Typical cost for hosting & website, emails, some tools, cloud applications like project / requirement / salary management, storage, collaboration, CRM, etc. – up to Rs. 50,000 to 70,000 per year depending on what options, products, etc. you select.
  4. Register your Trade Mark. Typical cost – Rs. 10,000 to Rs. 50,000 depending on what all classes you file in and what are the further steps / objections. Once in 10 years.
  5. You will incur expense on office like properly lease, electricity & salaries along with maintenance of current account, debit cards, internet banking, etc. – Depends on the city and location along with employee strength – Typically Rs. 75,000 to 200,000 per month for 3 to 5 employee company in Tier 2 & 3 cities.
  6. You will need to meet legal & compliance requirements for filing taxes, annual meetings, audit, lawyer, etc. via a Chartered Accountant – Rs. 40,000 per year.
  7. Digital marketing & sales activities will be required to be done regularly. Here are some major ones: Advertisements on Google AdWords, Promotions on Facebook, Twitter, LinkedIn, etc. – Typically Rs. 50,000 to 75,000 per year.
  8. You will need to maintain Internet, computers and other associated administrative activities. Expenses – Rs. 20,000 per month.
  9. Setup and regular filing / maintenance for EPF / EPS, Employee medical / life / accident / professional liability insurance will need to be done. Related expenses – Depends on the size of your firm and revenue. But a reasonable amount for a small company of 10 employees is about Rs. 15,000 to 25,000 per month. Some of these are optional depending on the size of your company.

My suggestion is to go with a Tier 2 or 3 city as your base and use digital medium to offer your services along with usage of online products on taxation, legal, compliance, audit, salaries, collaboration, sales, marketing, etc. to manage your company as much as possible with as small a footprint as possible in physical world to start your venture.

You can save a lot on office lease, physical sales & meetings, fuel, meeting customers, collaboration and managing data if you go digital.

Reach out to me for any discussions, help, suggestions on building a lean and low cost startup. Good luck ! !

Skills required for working as a Solution Architect

  • High level Architecture understanding about Clustering of Databases / Applications Servers / Operating Systems, Enterprise Architecture, Non Functional Requirements, Integrations, Various Enterprise Software Products & Frameworks
  • Concepts of Virtualization and Cloud platforms like AWS, Azure & Google Cloud
  • Integration – Tools, Frameworks & Products that can be used like ESB, Web-Services, etc.
  • Latest trends around Big Data, NoSQL, React, Mobility, Portals, ERP, Cloud, Search, Scanning, Analytics, Networking, etc.
  • Reading list of websites for updating oneself on various domains, trends & technologies
  • Documents, spreadsheets & presentations – Skills with MS-Word, MS-Excel, MS-PPT, Google Docs, HTML, Visio, Free / Open Source / Cloud based Enterprise Architecture & Digital Marketing Tools & MS Paint, etc.
  • Programming skills in at-least one front line full application programming language like Java, VB.Net, C#, etc. as a minimum & concepts of digital electronics & microprocessors
  • Database, Encryption & Computer Networking skills including SQL, PL/SQL, Indexing, Triggers, Procedures, TCP/IP, UDP, IPv6/4, SSL/HTTPS, Types of encryption, etc.
  • Basics of how to create requirement documents, proposals, estimations & statement of works
  • Soft skills for working & collaborating with various stakeholders from Top Management, HR, Finance, Legal, Operations to developer level and customers
  • Reporting skills around creation of document, status updates, etc.
  • Project Management & Testing overview, Tools, Processes & Products used for the same
  • Overview of Quality Frameworks like CMMi, ISO, etc.
  • Ability to learn things quickly, present them in concise, customer friendly ways & networking skills to support Sales & Project Managers
  • Basic understanding of RFP / RFI / RFQ & Consulting process

Part 1 for this here: What do Presales Consultants / Solution Architects do ?

Ideas on Innovation around Technology. We Thrive On Ideas. We are Learner Centered, Open Source & Digital Focused.