Testing for DevOps – How to have 10 deploys a day versus 10 emergencies a day

Posted on | by ccadmin

Ten deploys a day was the tech talk that started DevOps as an integrated practice for software development. However, if they simply pushed out code without checking it, then the story would have been different. Without testing the code, it would be more like 10 or more Disasters a day… A down production site would result in some serious consequences such as:

  • Loss of customers
  • Loss of confidence in the company
  • Loss of employment for all who are involved

You get the pictures. Though the use of automation is a key principle of DevOps, automation is also an efficient way to multiply human error. Without efficient testing of code, CICD would be meaningless. If you write code quickly and deploy it, BUT it’s broken code, well you defeated the purpose of having software rapidly deployed and updated. Testing of code is one of the most fundamental part of DevOps there is.

But I’m a great coder and I don’t make mistakes…(or whatever is your excuse)

Well, that may be the case, however as your software product grows and more people are touching a code, the likelyhood some form of human error (that may not be yours) will be introduced into the code. It’s bad enough when you haven’t worked on the code let’s say in a couple of weeks, you don’t remember what you did. You don’t want to be called back into work on a Friday because your code changes you created bombs out on a production system… Test for it so you can sleep soundly knowing that you have a safety net in place.

Writing tests is a fundamental practice.

The fundamentals – Vince Lombarti, who is know as the winningest coaches in football once went to the lockerroom of his players after they had a major loss from a previous season and said .“..This is a football…” Part of writing good code is writing checks or tests for your code. Unfortunately, many programmers don’t write tests for their code, especially web developers who are use to just writing and fixing it as it breaks. It has been my experience that the reason many programmers don’t test code is quite simply, they don’t know how. We’ll get you pointed the right direction.

What are the various test you can do for code?

There are hundreds of different tests you can do on software, some are appropriate and some are not appropriate for testing your software. Here are a list of some of the more common tests. For the purpose of testing in a DevOps environment, there are basically two categories of test you can do for code. I’ll put particular emphasis on what are the must have tests. We’ll cover the must have tests. We’ll touch on the other tests.

Functional Tests

These are the most basic test we need to have in place for Continuous Integration.

  • Unit testing
  • Integration testing
  • Systems testing
  • Acceptance testing

Non Functional Tests

These are also important tests but are not functional test (testing key functions in software) to have AFTER you have the core functional test in place. Here are a few non functional tests.

  • Security testing – find security holes and exploits in your code.
  • Performance testing – find out how quickly your code runs.
  • Stress testing – find out how much load your application can take before it degrades or breaks.

Behavioral driven testing

This is influenced by BDD – Behavior driven development which is at the core TDD – Test driven development where development of software is defined by very specific test cases. In the case of BDD, the tests are defined in terms of a user story. This affects how you would write unit tests and acceptance tests. Testing frameworks like cucumber support this kind of writing of user stories.

Unit testing

Every bit of function you write should (must) be unit tested. It should be the most basic test that is used with any code that goes into production.

Basic principles of unit testing.

Test the smallest testable part of an application. That is a unittest.

Write your function to handle a particular error – i.e. division by zero.

test fixture – what’s needed to perform a test

Write a test case – 1/0

Write an assertion. Unit test assertion functions are very simple, they are functions that call your function and compares the returned result or state or value. For instance if you function returns a numeric value, you may use an assertion like

functionx > 0

or

assert.greater(functionx, 0)

This is dependent on your framework or library as far as if it supports different conditions for testing assertion.

These tests are best done with a testing frameworks or assertion libaries. For java, there’s JUnit, Python uses unittest, Javascript can use JestJS, MochaJS, Jasmine.

You can read more about unit testing here.

Integration testing

This is also very important. What if the guys who are responsible for writing the database interface makes a change that the busness logic guy didn’t know about or the frontend guys didn’t know about the change in the functions for business logic. Your code justs breaks and that’s not good for your mental health. You can read more about integration testing here.

Acceptance testing

This is what your customer sees and interacts with. This is probably the most time consuming part of writing tests for. Never the less, if the customer is not able to press the “pay” button, you and your company are going to lose money…and that’s not good. Your going to catch this with a good automated testing approach. These days if your testing a web app or website, use selenium or a framework that drive selenium to test the web interface with clicks and data input values. You can record your test using the Selenium IDE in Firefox or Chrome. It will take some massaging of your recording to make a good automated acceptance test.

Depending on your testing strategy, your approach and framework you will use will vary. Regardless of your approach, I’ll always recommend unit testing your code.

Tying it in to DevOps

Continuous integration is a strategy and software testing being both a strategy for instance Behavior driven development with various frameworks as tactics such as cucumber as a framework that supports Behavior driven testing.

Without software testing and a sound testing methodology, you couldn’t have Continuous integration. That is the Holy Grail of DevOps.

I am open to any comments or suggestions to this article. Until then have a good day.

Book Review: Start with Why – by Simon Sinek

Posted on | by ccadmin

What does this have to do with (fill in the blank)?

In the IT field and especially if you are a tech, it’s easy to get in the “weeds” of what you do, and you forget why you do it. It’s when you forget your why that you often have writers block, make bad decisions and you have a hard time getting your ideas and projects from being accomplished.

The book Start With Why was written from a ted talk Simon Sinek did Start with Why.

He outlined as an example how an average computer company would sell computers, I’ll paraphrase:

Average computer company:

We’re GreatComputers company. We do 6 Billion dollars in sales. We make state of the art computers, that are user friendly and powerful.
Buy our computers.

This sounds like every other company and is not very compelling.

Here’s what Apple computer markets itself:

We’re Apple, and we’re against the status quo. We believe in empowering the individual. That’s why we make user friendly computers which are both easy to use and powerful using cutting edge technology. Do you want one?

You may notice, that the content itself hasn’t changed much except for the reason behind the content. This makes the message more compelling.

He gives several examples of this in his book such as Martin Luther King, The Wright Brothers and examples of failures of why such as Tivo and Wallmart that lost it’s why.

The key principle he outlines is the golden circle where its:

why=>how=>what

This golden circle model is based on how the human mind works. The limbic primative brain is where the feeling that drive our actions come from. It is where our hunches and intuitions come from. When people say “It just doesn’t feel right” or wordings to that extent, How and what comes from the outer parts of the brain, the Neo-cortex. This is where our language and logic comes from.
We may have all the intellectual understanding to do something, however if your heart is not in your work or you don’t have any kind of passion to do the work, then things like burnout, anxiety and other stress related problems will creep in. It’s better to do what you believe in than not. Your feelings drive your logic and actions, not the other way around.

Without your why, then how and what become meaningless exercises and your actions and efforts just become a commodity.

How do I use Why in DevOps? (or anywhere else)

The why is what’s behind the strategies that guide your groups and organization. It guides your tactics and helps you build trust in your group. They “buy in” you need to make things work. It is also important for understanding individually why you do those things. For instance, you might get push back from a particular individual in operations for deploying a new software package. You go and talk with him and find out he’s the one who gets all the pages and ends up fixing the new package every weekend. If your able to address his why that is “I don’t get any piece of mind when a new package is deployed” then you remove that resistance and you will get buy-in from him.

Also with a good why, when they buy the why, they will help figure out the technical details. Shared intelligence is better than being the smartest man in the room. If your the smartest man in the room, you are doing all the work. You don’t want to be a micromanager.

Everyone is a leader – Start by leading yourself

Everyone is leading someone. At the end of the day, you are leading yourself. Start with your own why and communicate with yourself why your doing whatever your doing. Then communicate this with everyone else you are working with. It makes it easier for others to work with you and follow along.

This is what I found useful from Start With Why and my main takeaways are:

  • Why – remember why your doing it.
  • Look for peoples whys.
  • Communicate your whys to other people.

Your mileage may vary.

Feel free to give me your comments and suggestions.

Until then, have a good day.

Blackbird – Privacy tool for securing your Windows PC

Posted on | by ccadmin

One of the cool things about using a package management system is you can look at a central repository to see if there are interesting and useful tools for making your life as a Systems Administrator easier. Even if your title isn’t a Sys Admin, you end up fixing your own machine anyways.

In the Chocolaty public repository, there is a package Blackbird which will secure your Windows machine for privacy and it does a lot. It removes not only the common privacy inhibiting updates, it also helps speed up your machine by turning off many services you don’t need.

What’s the big deal, I got nothing to hide?

Maybe you don’t and I’m not one to judge. The reality is, many big software companies indiscriminatly collect information through the Windows operating system. We don’t really know what they use this information they collected on us for. I like keeping my information private. Here is a perspective given by Alok Bhardwaj TEDx talk. His company also developed the Epic privacy browser.

This program does not only apply to Windows 10, but also older versions of Windows (8, 7 and Vista).

Isn’t removing Windows telemetry updates enough?

One would think so. According to the Blackbird site it does the following:

  • Removes Windows updates that enable tracking – 51 updates.
  • Disables services that talk to Microsoft – 23+ services.
  • Blocks IPs addresses used for Telemetry – 364 urls

That is a WHOLE lot of settings to tweek.

Installing Blackbird

Using Chocolaty.

In command line type:

PS C:> choco install blackbird

or

Download from here.

Running blackbird

Blackbird has a few command line options, here are a few useful ones:

-? Print all the options available.

-v Verbose mode. Displays additional info on all changes being made.

-s Silent mode. Skips any pauses

-scan Full system scan. Green is good/red is bad.

-std STD mode. Force removal of known spy tasks.

-l Fix network/LAN connectivity issues.

-i Fix Bluetooth pairing issues.

-bak Backup mode. Backup current system settings to an external file.

Blackbird needs to be run in Administrator mode.

Before making any changes on your Windows machine, make a backup:

PS C:\ blackbird -bak

I like seeing all the changes made and making sure networking works correctly. In Powershell in admin mode, type:

PS C:\ blackbird -v -l

I’d recommend checking for updates on blackbird periodically. Of course, with Chocolaty, this would be a snap.

PS C:\ choco update blackbird

If you cover both your local Windows machine with securing your browser activities with TOR Browser or any other privacy browser(i.e. epic) and aVPN service, you should have great privacy on your Windows machine.

Feel free to give comments and feedback.

Until then, have a good day.

The Elephant in the room – 3 key points of DevOps – Strategy, Tactics and Implemention

Posted on | by ccadmin

Elephant in the room?

When people talk about DevOps, most people don’t know what their talking about or usually they only know part of what is DevOps. The blind men describing an elephant is what comes to mind.

One blind man says he feels it’s like a rope. The second blind man said it’s like a thick branch of a tree. The third blind man said it’s a huge wall.

DevOps is the elephant in the room.

We tend to only “see” one part of the elephant and we view it from our own role or perspective.

For many people, the answer of what is DevOps tends to falls between these kinds of viewpoints:

  • To systems administrators, it’s about use of Automation techniques to manage many machines and deploy code (i.e.Ansible, Kubernetics).
  • To a developer, it is a consistent code development environment and automated integration system (i.e. Docker, Git).
  • To a project manager and stakeholders, it is an Agile development methodology for software and product development.

These answers are both right and wrong at the same time. This is where the understanding of Strategy and Tactics come into play. With this, you can recognize and understand the “elephant in the room” and how to use these tactics and strategies.

The Difference between Strategy and Tactics

The culture and management practices of DevOps can be looked at as the Strategy. The implementation DevOps in the form of tools and processes used are the Tactics. One is at a global level, the other is the “hands on” task(s). Strategies don’t change often. Tactics do change depending on the situation. It’s an important to understand these perspectives to understand what “DevOps” really is.

What is Strategy from the DevOps point of view?

Strategy is the plan to archive a long term goal.  DevOps may employ many different strategies.  Here are a few examples:

These strategies are NOT always going to be appropriate for your shop for a variety of reasons and may need to be modified or other alternative strategies may need to be used.   SCRUM vs Lean?  Up to you and your team.

What is Tactics from the DevOps point of view?

Tactics is the actual methodology of implementation of a strategy.  Depending on your own needs and circumstances, the tactics you chose depend on the strategy your seeking to fulfill. Here are some examples of of tactics:

  • Using a Git shared repository is a tactic for Continuous Integration .
  • Using  Kanban software and workflow is a tactic for Lean software development.
  • Using Ansible to setup standardized servers is the tactic for the “cattle not pets” strategy.
  • Having small team of five people consisting of people from Operations, Development and Quality Assurance is the tactic  for the two pizza team and break down silos strategy.

Tactics may change for different systems you manage.  For instance an MDM (Mobile Device Manager) platform would be the appropriate choice for managing Smartphones NOT Ansible or Chef and in this sort of environment, Docker may not be appropriate for development of the front end app.

Implementation is the key

Even after understating the Strategies of DevOps and developing the Tactics to implement DevOps, if people are not DOING the tasks to implement the tactics,  just doing what they always done, they don’t have a DevOps shop. People need to participate. The place to begin is with collaboration and creating an environment where there is trust and the people involved are actually carrying out the tactics as day to day activities. To get collaboration, people need to have ownership in the project and get recognition for their part. Create an environment where people are willing to take risks and are not afraid of failure. To do this, it’s important that failure is valued as part of the learning process instead of blaming people or finger pointing.  In fact back in the 70’s at IBM, Tom Watson Jr. recounts calling to his office a young executive made a $10 million dollar mistake. Expecting to be fired, the executive presented his letter of resignation. Tom Watson just shook his head:“You are certainly not leaving after we just gave you a $10 million education.” Learning from failure is part of IBM s culture, it should be part of every successful company s culture.

There is NO right way to do this

You can read books like The Phoenix Project or  The Goal (Which The Phoenix Project is based on) and apply the Theory of constraints, or  read any number of books written about DevOps.    However, the original talk they done on 2009 10+ Deploys Per Day , Allspaw and Hammond didn’t do something all that new as these concepts were done by others.  Lean manufacturing is where Kanban and Lean software development came from.   Site Reliability Engineering , which is in principle DevOps, was developed at Google around 2003, well before the Velocity 9 conference.  At Netflix,  they weren’t thinking about DevOps when they developed their “DevOps” shop.
I am of the opinion that you have to examine BOTH the STRATEGIES AND TACTICS of what worked at various organizations and what didn’t and focus on collaboration.  Building a collaborative environment that encourages ownership of a project, gives recognition and encourages people to take risks and not be afraid of failure is the key first step to creating a DevOps shop.  Doing this will make adopting  the “Elephant in the room” less intimidating and will put DevOps in your grasp.

Feel free to give me your comments and suggestions.

Until then, have a great day.

Do you Get Git? Git for source control and DevOps

Posted on | by ccadmin

Source control is one of the core strategies for DevOps since having developers and operations team members have access to each others processes require a copy of the code and configurations available to everyone involved.
Whether you write code or just have files you continually have to revise, we need to store our files and keep copies safe.
Git provides a solution for source control and collaboration. In fact one of the largest collaborative open source platforms GitHub is built on GIT. They were recently acquired by Microsoft for over 7.5 Billion dollars.

The Problem that GIT addresses.

When I was starting out and didn’t know any better (some people say I still don’t), I would make a copy and store it in a directory. I also may store a new file with a updated name like Original.txt. I would then have a copy named Originalv2.txt Originalv3.txt as I kept making changes and updates to my file.

However, this over time became quite messy to look at and confusing after a short time.

Further, if I wanted to share this group of files with someone, I would have to tar or zip the file up and send it.
On top of this, if I’m working with someone to make changes, they would send me a file and hopefully, I add it to the right place in the project…this gets very inefficient.
This is where source control comes in.
There are many different solutions such as subversion, mercurial, etc, we’ll focus on Git.

What can Git do?

Linus Torvalds and the developers of the Linux kernel needed to replace their proprietary source control software (BitKeeper) and he made a request to the community of developers to create a source control software that had the following requirements:

Open source and free
Distributed not centralized
Able to work offline

Git is Open Source and free. Enough is said, If you need to make changes to the code to the program, you can.  Further, there is no fee to pay for the software.  Support is available from a large and active community.

Distributed not centralized. Everyone has a copy of the files. If you need to see the code that is written or a configuration file, it’s right there.

Able to work offline.  Changes can be made locally and on a central repository. You can work offline with Git. Make changes to your files and commit them both offline and on the network. Many source control systems don’t allow for offline support.

You need source control as you write code. You may make a mistake and need to roll back. Your system may be running abnormally and your files may end up corrupted… but fear not! You made a backup of that program file…or did you… Maybe its an older backup that you need to recover your code from…how far back…

This is where Git is very useful to have. You can make a local revision backup of your program.

Rather than explaining how it works, go and download a copy of Git.

Installing Git

Get git here

or use Chocolaty

$ choco install git

Git supports a variety of ways to setup a repository.
Local, HTTP/HTTPS (smart with authentication and dumb),SSH/SCP
In this article, we’ll cover local.

Setup local repository for your own files

$ git init

then

$ git add file1 file2 file3 …

or

$ git *

then

$ git commit -m ‘initial commit’

There’s more

You can upload your files to a remote repository, clone a remote repository and have this used in a DevOps environment.  It works very well with Continuous Integration/Continuous Deployment where you can setup a tool like Jenkins to start processing the code out of your repository as soon as there is a change.

For more info on Git, download the book ProGit free in e-book format.

Suggestions and ideas for using Git.
Lastly, what good is knowing about a tool without knowing how you can use it?
Many people have found other ways to use Git besides for source control for coding.
Besides using Git for making copies of code on the repository, you can use it for making
backups. People have written articles, books and distributed it using Git.
In systems administration, you can keep critical systems files using Git
I use it for my Vagrantfiles and Ansible playbooks for my systems configurations.

Let me know your comments and suggestions you may have.

Until then, have a good day.