Author Archives: David Joo

Ansible Automation Platform – developer high-level workflow

Leave a reply

For a customer recently, I had to talk about with Ansible Automation 2.x, what is required to develop ansible playbooks.

Here is a high-level workflow diagram that I drew;

Ansible Automation Platform – developer high-level workflow

So what it is that… When you are writing a playbook and testing it, you need the following components:

  • Ansible IDE tool – my current favourite is VSCode, because there are so many nice extensions + Red Hat recently have released ansible extension
VSCode Ansible extension
  • Ansible-Core – the command line tool, the language and framework that makes up the foundational content before you bring in your customized content.
  • Ansible-Builder – to build execution environments
  • Ansible-navigator – to run, test playbooks with execution environments

If you haven’t built an execution environment, the very first thing that you need to do is to build an execution environment, as below:

4 files that you need to create are;

  • bindep.txt – Bindep is a tool for checking the presence of binary packages needed to use an application / library, so whatever is defined in this file will be installed.
  • requirement.txt – The python entry points to a Python requirements file for pip install -r …
  • requirement.yml – Outlines ansible collection requirements for galaxy to download and include into the execution environment.
  • execution-environment.yml – A definition file as an input and then outputs the build context necessary for creating an Execution Environment image

Detailed examples can be found in:
https://www.ansible.com/blog/introduction-to-ansible-builder
https://ansible-builder.readthedocs.io/en/latest/

Once the required execution environment is ready, it can be shared across your colleagues to enhance the collaboration experiences through consistencies.

Also, now you can start to develop an ansible playbook;

Finally, once you are happy with the playbook and the execution environment, it should be uploaded and managed in source management systems:

  • playbooks – Source Control Management Systems – e.g. github, gitlab….
  • EE image – e.g.) Automation hub, Quay.io, artifactory…

Then those can be properly leveraged by Ansible Automation Platform.

Ansible Automation Platform (AAP) 2.1 – released

Leave a reply

Last week finally, AAP 2.1 was released.
Here is the release note: https://access.redhat.com/documentation/en-us/red_hat_ansible_automation_platform/2.1/html/red_hat_ansible_automation_platform_release_notes/index
Here is a blog post from Red Hat: https://www.ansible.com/blog/introducing-red-hat-ansible-automation-platform-2.1

So to recap some of highlights are;

What’s included in AAP 2.1 – https://access.redhat.com/documentation/en-us/red_hat_ansible_automation_platform/2.1/html/red_hat_ansible_automation_platform_release_notes/platform-introduction#whats-included

Automation Mesh:
This is the newest addition to Ansible Automation Platform, and replaces the isolated nodes feature in 1.2. By combining automation execution environments in version 2.0 with automation mesh in version 2.1, the automation control plane and execution plane are fully decoupled, making it easier to scale automation across the globe. You can now run your automation as close to the source as possible, without being bound to running automation in a single data center. With automation mesh, you can create execution nodes right next to the source (for example, a branch office in Johannesburg, South Africa) while execution is deployed on our automation controller in Durham, NC.

Automation mesh adds:

  • Dynamic cluster capacity. You can increase the amount of execution capacity as you need it.
  • Global scalability. The execution plane is now resilient to network latency and connection interruptions and improves communications.
  • Secure automation. Bi-directional communication between execution nodes and control nodes that include full TLS authentication and end-to-end encryption. 

Satellite 6 and its partitions

Leave a reply

This is a note for my own to remember partitions and recommended sizing that I have been using. Below partition table was created to be inline with various security benchmarks. e.g. CIS/Essential 8

OS Partition - 75 - 80 G Required

/boot - 1GiB
/ - 15 GiB
/home - 10GiB
/tmp - 5GiB
/usr - 15 GiB
/var  - 5 GiB
/var/log - 10GiB
/var/log/audit - 10GiB
/var/tmp - 5GiB
swap - 2G
Satellite
Satellite - 650G - 1 TB required
/var/cache/pulp/ - 20 GiB
/var/lib/pulp/ - 480 GiB
/var/lib/mongodb/ - 60 GiB
/var/opt/rh/rh-postgresql12 - 20 GiB
/var/spool/squid/ - 15 GiB

NOTE: From Red Hat Satellite 6.10, Pulp 3 is being used, MongoDB will be deprecated and it will be consolidated to PostgreSQL.

So from 6.10, the recommendations will be to increase PostgreSQL to 45G and remove /var/lib/mongodb.

Uploading an Ansible execution environment to a private automation hub

Leave a reply

With Red Hat’s new Ansible Automation Platform 2.

Now you have to create an execution environment for you to run your automation.

If you are interested in knowing more about what’s Ansible Execution Environment? and how to build the execution environment please read more on;

  • What’s Ansible Execution Environment?
  • How to build the execution environment?
  • How to run Ansible automation with the new ansible-navigator?

To get into the topic now.

With the new Ansible Private Automation, Red Hat has incorporated a “container registry” into the Ansible Private Automation hub.

Ansible Automation Private Hub 2.0 – Container Registry

So how do we upload the execution environment that you built locally using “ansible-builder”?

Make sure you have your locally built image ready.

For this example, the image name is “my_first_ee_image“

(builder) # podman images
REPOSITORY                                                                         TAG         IMAGE ID      CREATED       SIZE
localhost/my_first_ee_image                                                        latest      18ee9d1f8d86  2 weeks ago   747 MB
<none>                                                                             <none>      f64cafba5f7b  2 weeks ago   740 MB
<none>                                                                             <none>      2ee39fed1806  2 weeks ago   747 MB
<none>                                                                             <none>      36ea822bf2b4  2 weeks ago   740 MB
quay.io/ansible/ansible-runner                                                     latest      a24b29574c26  2 weeks ago   725 MB
<none>                                                                             <none>      d5790b11bfe2  2 weeks ago   747 MB
<none>                                                                             <none>      20839e67474e  2 weeks ago   647 MB

1. Log in to your automation hub from CLI

(builder) # podman login -u=admin https://<FQDN or IP>/ --tls-verify=0
Password:
Login Succeeded!

Because, I am currently running my Private Automation Hub locally, I had to add “–tls-verify=0” or “–tls-verify=false” option.

2.  Create a repository structure + tag on the Private Automation Hub using the local repository structure.

(builder) # podman tag localhost/my_first_ee_image <FQDN/IP>/my_first_ee_image

(builder) # podman images
REPOSITORY                                                                         TAG         IMAGE ID      CREATED       SIZE
localhost/my_first_ee_image                                                        latest      18ee9d1f8d86  2 weeks ago   747 MB
<IP/FQDN>/my_first_ee_image                                                        latest      18ee9d1f8d86  2 weeks ago   747 MB
<none>                                                                             <none>      f64cafba5f7b  2 weeks ago   740 MB
<none>                                                                             <none>      2ee39fed1806  2 weeks ago   747 MB
<none>                                                                             <none>      36ea822bf2b4  2 weeks ago   740 MB

3. Upload your build EE to the private automation hub.

The format is that;

# podman push <Image ID> <URL>/<Image name>:<tag>

(builder) # podman push --tls-verify=false 18ee9d1f8d86 <FQDN/IP>/my_first_ee_image:latest
Getting image source signatures
Copying blob 32e86e0f9e53 done
Copying blob f47aeb60ec80 done
Copying blob c6ecf1ab50fb done
Copying blob ba176c23a887 done
Copying blob bb33f1f5d1b5 done
Copying blob f481c8dd5cb9 done
Copying blob 1ad9df8c4500 done
Copying blob 205c0028fa95 done
Copying blob 21adbb7c8fd5 done
Copying blob 3f69621a2c45 done
Copying blob b121b5075674 done
Copying blob 50de17c442cc done
Copying blob 39ac6dd69b36 done
Copying blob 9c229aeded24 done
Copying blob 2653d992f4ef done
Copying blob 48c89702f240 done
Copying blob 07805e10d0e1 done
Copying blob 5f70bf18a086 done
Copying blob 58e24711c0de done
Copying config 18ee9d1f8d done
Writing manifest to image destination
Storing signatures

4. Check it in UI;

Now, this image can be used in Ansible Automation Platform 2.x, as a preview;

How do I know whether “EE” is properly created and destroyed throughout AAP2’s automation execution?

Leave a reply

So in the previous series of articles, I have discussed what Ansible Execution Environment (EE) is, and how it is being consumed in the Automation Controller.

But really, how can I tell whether it is being ran or not?

Simple!
This can be validated by running “watch podman ps” on (a) execution node(s).

Below are 3 screenshots from moments of “before”, “during”, “after” a sample automation execution.

Command to run:

# su - awx
# watch podman ps

Before:

Before the automation execution

During:

During the automation execution

After:

After the automation execution

As you can see from the above, execution environment is dynamically spun up as a container and cleaned up right after the execution is completed.

To create a random password & also create a SHA512 hashed password

Leave a reply

While I work, I need to create random passwords as well as create SHA512 hashed password so it can be added into places like kickstart and so on.

To create a random password

# date +%s | sha512sum | base64 | head -c 32 ; echo</strong>
MjE2NmQwZDQwMWUzMzQ5Yjk2ZDliMGJj

To create a SHA512 hashed password

This is based on https://access.redhat.com/solutions/221403
However, with python3, “print” needs to be in “print ()” form.

# python3 -V</strong>
Python 3.9.2
# python3 -c 'import crypt,getpass; print (crypt.crypt(getpass.getpass()))'</strong>
Password:
$6$21eDal65kOhgvewf$EZLptcc4dOaC3/Fk8Js.seOslXTvt5JQF5TqPssft1tLCmoAZANMdGaqeZidGSVJ7X1RBIn/2xByfTvI9XAzg/

Ansible Core & contents

Leave a reply

Ansible Core is the command-line tool that is installed from either community repositories or the official Red Hat repositories for Ansible.

https://www.redhat.com/en/blog/ansible-101-ansible-beginners

With Ansible Automation Platform 2 release, few terminology changes were made.
One of those are, Ansible Engine as we know which included ansible binaries, modules are replaced with “Ansible-Core”.

Ansible Core is the foundational part of the Ansible Automation Platform.  It’s the command line tool, the language and framework that makes up the foundational content before you bring in your customized content.

The main differences between ansible-engine and ansible-core are “contents” e.g. modules and plugins.

Ansible Core only comes with limited number of contents.
(Number of ansible modules comparison between ansible 2.9 vs 2.11 can be found here.)

By moving contents out of the ansible-core, this provides following benefits:

  • Agility
    – Currently ansible contents are being developed and managed by Open Source Communities, partners and Red Hat. Now modules can be updated and managed through developers-driven and ansible-independent schedules.
  • A lean & Purpose driven execution environment
    – By only incorporating required plugins and modules, it bring the focus back into the users’ automation environment, rather than overloading with unnecessary contents.

As you can guess, this change was another foundation for Ansible Automation Execution Environment a.k.a ansible EE.

Ansible module numbers difference between ansible 2.9 vs ansible 2.11

Leave a reply

From, Red Hat’s Ansible Automation platform 2, one of the big changes is that the “ansible engine” is now changed to the “ansible core”.

With the change, the biggest change is that number of ansible modules that were shipped with ansible has dropped significantly.

This change was made to decouple ansible binary with modules.

Ansible modules development is mainly driven by;

  • Community
  • Red Hat & partners

This decoupling would provide shorter and community-driven release schedules, which usually doesn’t sync with actual ansible binary release.

More details Ansible Core change details can be found in HERE.

However, coming back to the original question, so how many ansible modules are included in ansible 2.9 and ansible 2.11?

N.B. This is based on my current test environments, where I have ansible 2.9.22 and 2.11.1. Also two environments’ installation methods are different. 

2.9.22 – RPM -> 3232 modules

2.11.1 – venv/pip -> 71 modules

2.12.0-2 – RPM -> 70 modules

ansible 2.9.22

[root@insights-client2 modules]# rpm -qa |grep ansible
ansible-2.9.22-1.el7ae.noarch
[root@insights-client2 modules]# pwd
/usr/lib/python2.7/site-packages/ansible/modules
[root@insights-client2 modules]# ls -lR |grep ".py" | awk '{print $9}' |grep -v ^_ |grep -v "pyc" |grep -v "pyo" |wc -l
3232

ansible 2.11.1

(py36-venv) [djoo@insights-client2 modules]$ ansible --version
[DEPRECATION WARNING]: Ansible will require Python 3.8 or newer on the controller starting with Ansible 2.12. Current version: 3.6.12
(default, Sep 15 2020, 12:49:50) [GCC 4.8.5 20150623 (Red Hat 4.8.5-37)]. This feature will be removed from ansible-core in version 2.12.
Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
ansible [core 2.11.1]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/djoo/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/djoo/py36-venv/lib64/python3.6/site-packages/ansible
  ansible collection location = /home/djoo/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/djoo/py36-venv/bin/ansible
  python version = 3.6.12 (default, Sep 15 2020, 12:49:50) [GCC 4.8.5 20150623 (Red Hat 4.8.5-37)]
  jinja version = 3.0.1
  libyaml = True
(py36-venv) [djoo@insights-client2 modules]$ pwd
/home/djoo/py36-venv/lib/python3.6/site-packages/ansible/modules
(py36-venv) [djoo@insights-client2 modules]$ ls -lR |grep ".py" | awk '{print $9}' |grep -v ^_ |grep -v "pyc" |grep -v "pyo" |wc -l
71

ansible 2.12.0.2

[root@ip-172-31-14-126 modules]# rpm -qa |grep ansible-core
ansible-core-2.12.0-2.el8ap.x86_64
[root@ip-172-31-14-126 modules]# pwd
/usr/lib/python3.8/site-packages/ansible/modules
[root@ip-172-31-14-126 modules]# ls -lR |grep ".py" | awk '{print $9}' |grep -v ^_ |grep -v "pyc" |grep -v "pyo" |wc -l
70

Ansible Automation Platform 2

Leave a reply

For the last couple of days, Red Hat just had AnsibleFest 2021. (https://www.ansible.com/ansiblefest)

Ansible Blog | Ansible.com | AnsibleFest

The biggest announcement would be on the Ansible Automation Platform 2.

Last July, there was a sneak preview + early access program for Ansible Automation Platform 2.0. (Link)

N.B. This is an “early access” program, which means?

Early access means that any Red Hat Ansible Automation Platform subscriber has the ability to download, install, and file support cases against this newly released 2.0 version of the product. Because there are additional core features and functionality that are slated for the 2.1 release later this year, the formal marketing launch for both 2.0 and 2.1 versions will happen later this year at AnsibleFest. Therefore, many of the typical resources (such as documentation, blogs, etc.) will only be made available on the Red Hat Customer Portal until formal launch at AnsibleFest.

So with the release of Ansible Automation Platform 2.1 in later 2021, Ansible Automation Platform 2 will be properly GA’ed.

However, in this article, I am going to focus on three main announcements:

1. Ansible Tower and Ansible Engine are no more.

=> Its replaced with Red Hat Ansible Automation Platform.

More details to be followed below.

2. Ansible Core – “Batteries are not included”

Ansible engine is now replaced with a component from Red Hat Ansible Automation Platform called “ansible-core”

Different to the Ansible Engine, the “ansible-core” will only include a limited number of core ansible modules.
(Number of ansible modules comparison between ansible 2.9 vs 2.11 can be found here.)

It seems like the changes were brought in for two reasons:

  • To provide agility in ansible module development.
  • To provide a lean ansible execution environment to end-users/developers.

    More information will be covered in a separate blog HERE.

3. Ansible Tower is split into smaller bits and utilises containers.

With the announcement of NO MORE ANSIBLE TOWER, the detail is that the Ansible tower is split into two separate components.

As the above shows, the Ansible Tower was in a single monolithic architecture. This works great. However, when there are multiple organizations/teams with multiple python virtual environments requirements, it started to get complicated really quickly. 

To address the above, Red Hat has replaced the execution/virtual environments with “Execution Environment”.

The execution environment is a container with various required components.

More information can be found HERE.

The rest of WebUI/API/RBAC/Workflows and Audit components are grouped into “Control Plane”/“Automation Controller”.

4. Red Hat Ansible Automation Platform has a lot more features/components

Red Hat Ansible Automation Platform features/components

Colour boxed ones are new components and features brought into the Red Hat Ansible Automation Platform.

* Ansible Platform Operator
– Red Hat Ansible Automation Platform is available on the OpenShift Container Platform as an Operator. 

This makes installation, operation tasks such as upgrade, easy. Also, provide the high availability capability automatically.

* Automation Controller & Automation execution Environments

As explained above, these two components replaced the old “Ansible Tower”.

* Ansible-Builder

This is a new component that enables an ansible content creator to build a custom/purpose-fit ansible automation execution environment (a.k.a. ansible EE).

* Ansible-Navigator

This is another new command-line component added to enhance ansible content creator experiences. With new the ansible EE, ansible-navigator should be used as a replacement for all too familiar “ansible” and “ansible-playbook”.

This is one tool that you can run, debug even introspect ansible EEs.

This will be covered in another article, HERE.

* REST of components

So until now, Red Hat Ansible had main 2 components as #1 suggests. On top of that there were few additions to that;
 * Automation Services Catalog – Service Catalog as a SaaS service on https://console.redhat.com/ansible/catalog/products
 * Ansible Content Collection – Red Hat and partners co-developed and co-supported ansible contents available https://console.redhat.com/ansible/automation-hub.
There are currently 102 partners and Red Hat have contributed content.
 * Automation hub – a single location where public and private ansible content collections will be hosted.
 * Red Hat Insights for Ansible Automation Platform – where it provided overall information of an organization’s Ansible automation platform usage as a dashboard.

Definitely, with the new version, the above features got richer. 

The Journey Begins

Leave a reply

Good company in a journey makes the way seem shorter. — Izaak Walton

post

I have been lazy in a way that gave up on having a separate blogs then moved to Korean blog site, tistory. Then I realised that, Tistory may not be friendly for people other than Korean.

So I am giving another attempt in maintaining a blog.

I hope this time it will be different.