All posts by Neil Harwani

Interested in movies, music, history, computer science, software, engineering, management and technology
Culture

Information Technology culture across company categories – Part 1

Leave a comment

Products:

  • High on collaboration
  • Cross department deliverables
  • Focus on specific areas like products, goals and such
  • They typically build their own strategy
  • Various sub-cultures like entrepreneurial, open source, hybrid and proprietary may exist within this
  • Controlled offerings for customers
  • Innovation & creativity is valued highly
  • R&D is important here
  • Flexibility in terms of what employees can do beyond the organization
  • Flat or semi-structured hierarchies mostly
  • Inventing a new category, offering, product, culture is possible here

Services:

  • No set boundaries on technologies, products & frameworks at times
  • Customer, partner & product ecosystems more or less drive the strategy & evolution
  • Standardization of delivery & facilities for employees
  • Silos & compartments within company by design and reason
  • Hierarchical & matrix way of reporting and work with management
  • Typically fixed set of roles & responsibilities for employees
  • Process plays an important role here

Startups & mid-sized companies:

  • No set hierarchy for interactions in the company – flat or semi-structured hierarchy
  • Focus on cutting edge domains and technologies at times
  • Evolves & builds strategy with the market & customers – Flexible in this aspect
  • Multi-tasking roles & responsibilities with potentially high growth and learning or relatively quick failure

Technology work in Non Technology product companies:

  • Focus on implementations of full projects & vendor, partner, product offerings mapped to business outcomes
  • Co-ordination with stakeholders within and outside the organization plays an important role here
  • Maintenance & support, evaluation of products & technologies, return on investment takes up a large amount of time

#culture

Email me: Neil@HarwaniSystems.in

Education

PhD/FPM/EFPM, PGDM, MBA, MSc, MTech – “The Crystal Maze”​ demystified – Part 1

Leave a comment

Here is my attempt to demystify – “The Crystal Maze” of Indian Master’s degree / PGDM / EPGDM / MBA / Executive MBA & PhD / FPM / EFPM. “The Crystal Maze” was a TV series as described here: https://en.wikipedia.org/wiki/The_Crystal_Maze – the programme focused on teams of contestants, a mixed group of men and women, attempting a range of challenges to earn time required to help them complete one final challenge, which if completed successfully earns them a prize. – From Wikipedia page of the topic.

Many students have questions around PGDM / EPGDM / MBA / MTech / PhD / FPM / EFPM and similar depending on their aims. Here are some notes for the questions that will drive you towards the answers:

  1. Where do you go for checking validity of PGDM / EPGDM in Full Time / Part Time / Online / Distance Learning mode – AICTE, UGC Distance Education Bureau websites
  2. Where do you go to check if your PGDM is equivalent to a standard MBA – AIU website
  3. Where do you go to check if your FPM / EFPM is equivalent to a PhD – AIU website
  4. Where do you go to check validity of degrees from central universities / autonomous universities & institutions of eminence – UGC, AICTE, AIU & MHRD websites
  5. Where do you check rankings of universities & institutes in India – NIRF
  6. How do you do a PhD if your PGDM is not equivalent to MBA – Either go to IIMs, IITs, Government institutions & universities which are autonomous / institutions of eminence or universities in the advanced economies
  7. What are the accreditations you should be checking – NAAC & NBA
  8. What are the foreign accreditations that Indian institutes & universities participate in – QS world rankings, Times Higher Education rankings, there are more as well
  9. What is triple crown – AACSB, AMBA, EQUIS – These are three accreditations which if a college / university / institute has they are called triple crown accredited

List of websites referred above:

  1. UGC – https://www.ugc.ac.in/
  2. MHRD – https://www.education.gov.in/en
  3. AICTE – https://www.aicte-india.org/
  4. AIU – https://www.aiu.ac.in/
  5. Triple crown – https://en.wikipedia.org/wiki/Triple_accreditation
  6. NIRF – https://www.nirfindia.org/Home
  7. NAAC – http://www.naac.gov.in/
  8. NBA – https://www.nbaind.org/
  9. QS world rankings – https://www.topuniversities.com/
  10. UGC DEB – https://deb.ugc.ac.in/
  11. Times Higher Education Rankings – https://www.timeshighereducation.com/world-university-rankings

Email me: Neil@HarwaniSystems.in

Experience

Word Cloud of my twenty years

Leave a comment

I started work by programming few websites around the year 2000 approximately. Since then on and off depending on my role or work I have been programming, doing business analysis, building solutions and recently in last few years have been working in education & customer success.

Below is a word cloud which summarizes my experience and learnings, I hope you find it useful. I will be updating it regularly every few months. This is version 0.1 and needs many more updates which will come iteratively

Email me at: Neil@HarwaniSystems.in

DevOps

DevOps in Networked & Technology infused world

Leave a comment

My original article below, co-authored with Prof. Nityesh Bhatt. An abridged & modified version of this was published in a leading magazine.

Advice to youngsters, be your own publishing house – you have Amazon, LinkedIn, your own blog & website, multiple other good platforms, peer reviewed open access & top journals, don’t depend on others for publishing your work.  Message, context, grammar & meaning may at times be lost when we publish via third party systems.

Authors: Prof. Nityesh Bhatt & Neil Harwani (Visiting Faculty) – Institute of Management, Nirma University, Ahmedabad 

Date: 18th June 2021 

What is DevOps?

  • We are in a world that is providing services via eBanking, eHealth, eGovernment, eCommerce and many other electronic interfaces and systems. Things are getting digitized at a rapid pace. On one side, the common man uses these digital services and requires changes to meet their dynamic requirements for their daily work with software and technology companies managing these systems. But there’s another angle to all these electronic systems which is the developer community. 
  • Developer community & software project management evolved from the past by using concepts of waterfall in software development lifecycle, pair programming, iterative development and so on over the last few years and decades. Various programming styles like structured & object oriented evolved along with these methods. 
  • Overtime, these older methodologies became inefficient at delivering complex, time bound and customer outcome oriented products that changed over the course of time during development due to changing market needs. With this came AGILE & DevOps over the last few years. 
  • AGILE is a methodology where product specifications evolve over time and development happens in small 1 to 2 week sprints, customers are shown demos at the end of each sprint and sometimes everyday. AGILE gives developers, product owners and management a flexible approach to build software products which can evolve over time and are aligned to customer needs with course corrections possible at any point of time during project lifecycle. Primary keywords that are used in AGILE are: Scrum masters, Developers, Product owner, Sprints and so on.  
  • Overtime, limitations of AGILE were observed and then DevOps was created to improve the culture, deliverables and duration for project / product delivery. 
  • Requirements for collaboration & efficiency for geographically distributed teams became a necessity due to big improvements in network technology. This led to DevOps and rapid delivery becoming the norm of many software development projects.
  • Over the last 10 to 15 years, traditional software development with Waterfall methodology has rapidly moved to AGILE, iterative and hybrid AGILE software development methodologies. 
  • This shift was necessitated due to a large number of failures in software projects, cost overruns, customers not getting the product they want and such problems. 
  • AGILE has brought with it it’s own challenges in terms of weekly or two-weekly demos & deliveries. 
  • Developers can no longer wait for the operations team to ready the environment in a few days and weeks, they need continuous delivery, continuous testing, weekly demos, light weight processes for design, development, testing, documentation, packaging & release – all in a cloud environment at times. 
  • DevOps is a methodology where we combine Software Development & Operations and create a seamless methodology which works well in conjunction with AGILE software processes. Development team takes responsibility both for development as well as operations which significantly increases the throughput.
  • DevOps spans across Coding, Building, Testing, Packaging, Releasing, Configuring, Monitoring with cross-functional teams working on all these areas 
  • Timeline for Agile practices is given here at Agile Alliance: https://www.agilealliance.org/agile101/practices-timeline/
  • Timeline for DevOps as given on TechTarget.com: https://whatis.techtarget.com/reference/The-history-of-DevOps-A-visual-timeline 
  • Importance of DevOps to developers: https://www.statista.com/statistics/1127211/devops-importance-to-scaling-software-development/ 

UseCases for DevOps and rapid, customer focussed, sprint based deliveries: 

  • Smart cars – Car apps and navigation need rapid upgrades & security patches
  • Banking & Financial industry applications – Loan & Insurance processes regularly change which necessitate DevOps usage 
  • Smart electronics – Application philosophies from over the air to satellite to over the top to hybrid over the top with ecosystem apps change over few years and decades. Applications using them and their billing / user interfaces regularly get updated. 
  • Mobile phone applications – Every few months there are updates to operating systems, applications and overall ecosystem for regulatory, security & privacy reasons 
  • Digital manufacturing & 3D manufacturing require changes in assembly & production lines to cater to large numbers of products – Wide variety of products now need to be made from automated assembly lines & 3D printers. These need reprogramming at regular intervals
  • NBFCs – They have changes especially in micro-credit market in terms of coverage, limits and which segment to cater to 
  • Online education – From pre-pandemic to post-pandemic there has been change in acceptance, delivery & ecosystem of education. Online education is now acceptable for large parts of the population as well as universities & institutes. Quick changes in courses, delivery & methodologies can be facilitated by projects undertaken via AGILE & DevOps thinking. 

Why DevOps in Agile, Cloud, Analytics and 5G world?

DevOps is necessitated due to following reasons:

  • Cloud environment – Availability & need of virtual instances which can be scaled up and scaled down in seconds / minutes to meet varying demands of customers from different domains
  • Dynamic internet, cloud, analytics & 5G driven world where things are getting streamed, changes & integrations are common place
  • Due to the dynamic nature of the business environment, changes need to be deployed quickly. This is where AGILE & DevOps comes into play for rapid releases on bi-weekly or monthly basis.
  • Today, new application upgrades in Telecom, Analytics, Cloud, Manufacturing, Cars are deployed on a monthly basis at times. They necessitate a faster, relatively failure proof system of development, testing & release management.
  • Containerization and orchestration of containers in the cloud ecosystem provides these options and this in turn allows us to enable DevOps work at speed.
  • Requirements & design are discussed over calls and stored in tools like JIRA with development happening in DevOps way via containers. This in conjunction with weekly or bi-weekly demos & meetings allow regular course corrections.
  • Developers can’t wait for multiple team members to setup their environment or applications. Containerized environments and build tools with CI/CD pipelines allow them to do all this seamlessly.
  • Same concept applies to all the network upgrades in the 5G world, applications over networks, software defined radios / networks and so on. These necessitate rapid development infused with seamless operations for managing environments. 

https://assets.kpmg/content/dam/kpmg/be/pdf/2019/11/agile-transformation.pdf – As per the article from KPMG: “Agile Transformation From Agile experiments to operating model transformation: How do you compare to others?”, main drivers for Agile which in turn percolate to DevOps too:

  • Faster delivery for changing customer needs
  • Flexibility
  • IT & Business alignment 
  • Digital agenda goals attainment
  • Customer satisfaction 

Technology infused world 

  • We live in a technology infused world where rapid changes happen around us 
  • Every few years network technology changes, examples: 2G to 3G to 4G to 5G, Changes in app & billing ecosystem, Security updates, Software defined networks, Software defined storage, In-prem to cloud to hybrid cloud
  • Similar changes happen in navigation devices, mobile apps, smart cars, banking apps, NBFCs and such 
  • DevOps & Agile facilitate rapid development & deployment enabling businesses to achieve their goals in terms of upgrades and features in an environment that is contained, controlled and process driven with clear responsibilities and roles
  • Earlier in Waterfall compared to AGILE & DevOps now, there were no single set of atomic responsibilities for developers or product owners / managers. With introduction of these two philosophies, ownership and overall structure of technology products and their maintenance cycle (full life cycle) have seen quantum shift in terms of thinking 

DevOps, DevSecOps and MLOps

  • DevOps – Combines Development & Operations
  • DevSecOps – Combines Development, Security & Operations 
  • MLOps – Combines Machine Learning pipelines & Operations to productionize it in the same way as is done in DevOps 
  • Lot of innovation in terms of processes and tools is happening in the DevOps world especially around security, testing & continuous integration / deployment 
  • Multiple integrations for various aspects of DevOps span across products & tools in client enterprise ecosystem enabling most if not all technical & functional aspects of project / product delivery / customer needs to function in a quick and efficient manner 
  • Improvements in both AGILE like Scaled Agile & in DevOps like MLOps have been happening over the last few years in conjunction with all the improvements and new launches in Cloud ecosystem 
  • For development teams, initial learning curve of DevOps & AGILE could be steep but as they get settled into these newer ways of working, processes & systems around these concepts provide a cover which customers like, developers enjoy, business owners prefer and overall management teams get better outcomes from 

Popular tools, frameworks, products, toolchains & options:

  • Docker – Container environment 
  • Kubernetes – Orchestration system for containers
  • JIRA – AGILE / DevOps process tool 
  • Ansible, Terraform – Automation related 
  • Git – Source code repository
  • Jenkins – Continuous integration / Continuous deployment related 
  • Splunk – Data platform 
  • Bamboo – Continuous integration / Continuous deployment related 
  • Automated security, performance & testing tools / products – Katalon 
  • ELK – Search and analytics engine
  • Utilities like MAVEN, Gradle and such 
  • Monitoring tools like Nagios, NewRelic and such 
  • AWS, Azure & GCP – Cloud

References:

Email me: Neil@TechAndTrain.com

Agile, DevOps

Reasons to adopt AGILE & DevOps

Leave a comment

DevOps:

  • Developers need environments to be readied, recycled, shared, rebuilt in short period of time with least amount of lost time, control and additional jumps
  • Fused teams of developers, system admins where responsibilities are more or less completed via multi-tasking & multi-skilling by each member

AGILE:

  • Early feedback for product owners, developers & testers
  • Continuous feedback / demo driven development
  • Required documentation (Not less not more) via specialized tools
  • 1 to 4 week sprints giving out tangible outputs that can be demoed
  • Course corrections possible in the middle of the project / product life cycle

Tools / products / technologies for AGILE & DevOps:

  • JIRA and similar
  • Jenkins and similar
  • Containers, Kubernetes and similar
  • Git and similar
  • Cloud ecosystems
  • Ansible, Terraform and similar automation
  • Video conferencing tools
  • Monitoring & log products
  • Testing tools
  • Ticketing & Chaos engineering tools
  • Reporting tools
  • And more

References: Google Search, https://victorops.com/ 🙂

Reach me at: Neil@TechAndTrain.com

Analytics, Data Science

Four waves of Artificial Intelligence & Machine Learning

Leave a comment

While teaching students in two different courses (AIML & “Data Science and Analysis”), there was a requirement to categorize historical AI & ML along with it’s interface with Data Science.

To start: AI is the superset, ML is a subset of AI, Neural Networks (Deep Learning) are specialized subsets of ML.

Below is a categorization of AIML across four waves and it’s interface with Data Science:

Wave 1:

Concepts: Traditional topics like state space search, heuristics, knowledge representation, expert systems, fuzzy logic, problem solving languages and such.

UseCases: Think a small basic robot moving through your home and taking decisions on avoiding obstacles.

Wave 2:

Concepts: Standard algorithms built on top of Regression, Statistics, Algebra, Probability, Calculus and such – Classification, Decision Trees, Association Mining, Clustering, Ensemble methods, Random Forest, SVM and so on. NLP, Computer vision, scanning solutions, advanced search and such areas also evolved here in parallel or with the help of these algorithms.

UseCases: Spam detection, Decision making, Co-related variables related predictions, Prescriptive Analytics and so on.

Wave 3:

Concepts: Replicating human / animal brain. Neural Networks. Storing and managing very large amount of data (structured & un-structured)

UseCases: BigData, Self driving cars, Image recognition, Complex reasoning, Medical diagnosis, Chat bots, Personal assistants, potentially unlimited usecases interfacing with all usecases across AIML & Data Science.

Wave 4:

Concepts & UseCases: Explanability, Interpretability – Understanding the complexity of artificial intelligence & machine learning models. UI & Low code driven AIML (Neural Networks), one shot learning, hardware optimized AIML. Deep Learning. BERT and newer context driven algorithms also are in this area, Natural Language Generation is another area here.

Where does Data Science interface with AIML:

  • Non structured data analysis
  • Natural language generation
  • Sentiment analysis
  • Use of standard algorithms to analyse structured data
  • Building insights & making predictions / prescriptions and so on

Email me: Neil@TechAndTrain.com

Culture

Traits of a good Information Technology company – Part 1

Leave a comment
  1. Promote merit and people with soft and hard skills both not just technology or soft skills alone
  2. Get them to attend trainings from HBR, MIT Sloan & various business schools from across the world regularly
  3. Setup a culture of knowledge sharing, collaboration & team work
  4. Plan & design all projects / initiatives in advance with spirit not lip service
  5. Setup accountability & responsibility
  6. For top management, managers, architects & leaders “Practice what you preach”
  7. Inculcate culture of cyber security, discussion & respect
  8. Keep people busy with genuine work
  9. Respond to people when they reach you, ignoring and tolerating problems for long will not solve the problems and snowball into bigger problems
  10. Few bad leaders will destroy culture across the organization rapidly
  11. Transparency & explaining WHY helps when you are a leader or manager
  12. Operational level problems over time convert into strategy level problems, fix them when they start. In the same way, strategic problems convert into operational problems quickly. Cater to both of these regularly
  13. Think design & systems thinking
  14. Plan and evaluate the future regularly – if you don’t invest in the future now, your path will derail from future when it should have come to you
  15. Have tie-ups with top institutes for further education / training of employees
  16. Give time off, leaves & hobby time to employees – they are not robots. People have people problems, robots too have breakdowns
  17. Actions have consequences, repeated violations have more consequences. Everybody needs to understand this
  18. Don’t judge the book by it’s cover – take 3-6 months of regular interactions to evaluate an employee. Instant judgements don’t help
  19. Awards, limelight & likes don’t always determine the worth of a person
  20. Anything can be achieved anyhow is a dangerous concept, companies that promote this beyond the legal or fair boundaries are not the right places to work – We are not at war, this is corporate world not World War 1/2
  21. Culture eats strategy for breakfast – this is true
  22. Promote, remove & reskill right set of people who have all stakeholders’ interest in mind, stakeholders should include employees & customers both
  23. Give regular & timely feedback at-least once a quarter, be open to discussions & feedback yourself as manager
  24. In internet and social media world, you cant hide your problems as an organization, they will bounce back with time most of the times. These can only be solved by discussion & transparency
  25. Shallowness, spam, power play, sycophancy & such can be detected easily, don’t promote it. Authenticity & transparency helps
  26. Sincerity, transparency & authenticity most of the times gives you the same in return, remember that
  27. Revenue is not everything, other things like strategy, culture, stakeholders matter equally
  28. Practice inclusivity and enablement of all sections of employees rather than only one set based on any bias
  29. Understand why and how biases are formed and correct them with right intervention

Email me: Neil@TechAndTrain.com

Pandemic

What all has changed post COVID-19?

Leave a comment

In the last one year, many things have changed for us. Here is a list of areas in business & life that have changed as observed by me & students who were attending the class for subject of BigData in MBA.

1. Digital payments now have wide acceptance 

2. Work & study from home is now a reality, locations matter a lot less now

3. Cyber security is now at the fore-front

4. Digitization of everything – Though we need to be careful of what, how and when we digitize

5. Mental & overall health for people is now a priority 

6. OTT platforms are now providing opportunities to various type of actors, film makers and others beyond the traditional ecosystems of cinemas & multiplexes

7. True democratization of knowledge / information & opportunity is happening, if you want to showcase your talent – internet & digital gives you that space 

8. Healthcare services are now online / tele-medicine is a reality 

9. Pharma companies are now collaborating in a big way 

10. Public health is now a priority for governments 

11. Digital innovation in physical businesses is now at the fore-front 

12. Online learning is widely accepted not just by certificate providers but by universities as well

13. Hiring & business models are changing 

14. RegTech is now mainstream

15. Local shops / stores / super-markets are now converging and getting a voice on various online app ecosystems / platforms – should the local stores act as small warehouses and deliver via app ecosystems, this could be the new model 

16. Newspapers / magazines are going for online subscription based models and shifting away from paper based publishing

17. App ecosystems have changed rapidly to accommodate alternative business models of deliver anything rather than specific things

18. Hotels, airlines & restaurants are looking at alternative solutions to problems that they faced in pandemics

This is a short compilation of changes that we see around us. It is not the strongest who survive, it is the ones who evolve and change that survive

What else do you think will change in the near term?

Email me: Neil@TechAndTrain.com

Uncategorized

Kubernetes topics to Search & Learn

Leave a comment
  • Pods
  • Containers
  • Registry
  • Master node
  • Secondary nodes
  • Control plane
  • Namespace
  • Ingress
  • Jaeger
  • Prometheus
  • Grafana
  • Kubectl
  • Docker
  • Kubernetes API
  • MiniKube
  • Logging
  • Kubeadm
  • Stateful vs. Stateless pods
  • Configurations
  • Yamls
  • Helm
  • Ansible
  • Operator
  • Metrics & Monitoring
  • Objects
  • Persistent volume
  • Persistent volume claim
  • Clusters
  • Auto cleanup
  • Kube proxy
  • Scheduler
  • Kubelet
  • Controller manager
  • api-server
  • Administration
  • Extensions & plugins
  • Patterns for Kubernetes
  • Authentication
  • Authorization
  • https://kubernetes.io/docs/home/

#kubernetes #clusters #orchestration #cloud

Email me: Neil@TechAndTrain.com

Visit my creations:

  • www.TechAndTrain.com
  • www.QandA.in
  • www.TechTower.in
Low Code

Best practices for low code development

Leave a comment

Best practices for low code development:

  • Finalize the domain / entity model early
  • Check the capabilities of platform before writing code for any feature
  • Learn XPath
  • Try to have daily builds, daily demos, daily development goals with short sprints – comes from agile world
  • Learn to use the CI/CD and Code management tools of the platform
  • Avoid deep linking URLs, avoid page URL altogether if possible
  • Check security / roles / entity access settings regularly
  • Integrate using in-built tools rather than custom code
  • For features like audit, associations, validations try to work at the entity level to start with. Incase you are unable to manage them their then move to workflow level validations
  • Understand the limitations of platform and then make appropriate suggestions
  • Old components, modules, etc. building up in apps should be cleared regularly
  • Check the app stores of the platform for modules that are already available before you build one
  • Check added libraries regularly for unused or deprecated / insecure versions
  • Scan uploaded files before using
  • Try to use architecture / design principles like SOLID in terms of singular responsibility and so on
  • Think of SSL, containerization, cloud, kubernetes, automation testing, logging, user interface & licensing in advance and not as a reaction later

Low code has arrived !!!