How to Deploy vRA 8.0.1 while dealing with the Built-in containers root password expiration, preventing installations for vRealize Automation 8.0 and 8.0.1

Let’s get into it right away.

A few weeks ago the 90 days account expiry from vRealize Automation 8.0 and 8.0.1 GA releases has been exceeded for both the Postgres and Orchestrator services which runs today as Kubernetes pods.

This issue is resolved in vRealize Automation 8.1 which is soon to be released as of the writing of this post. ( Generally available in 1H20 ).

This issue is also resolved in Cumulative Update for vRealize Automation 8.0.1 HF1/HF2 so if you already installed the HF1 patch a while ago and before the account expiry, you have nothing to worry about.

But what about existing deployments that was not updated with HF1 or HF2 as of yet or net new deployments of vRealize Automation 8.0/8.0.1 and how they may be impacted by this issue. In this blog I address those scenarios in terms of what needs to be done to continue benefiting from everything the automation solution have to offer today and/or have a successful deployment when you do choose to deploy the vRealize Automation 8.0.1 solution until vRealize Automation 8.1 is released then you really don’t have to worry about any of this.

So let’s get started eh!.

Existing Deployments

For existing vRA 8.0 or 8.0.1 customers with active working instances, you have two options before you can reboot the appliance or restart the vRA services:

Option 1

Apply the workaround mentioned in KB 78235 and stay at vRA 8.0.1.

Scenario 1 : vRealize Automation 8.0/8.0.1 is up and running

  1. SSH into each of the nodes
  2. Execute vracli cluster exec -- bash -c 'echo -e "FROM vco_private:latest\nRUN sed -i s/root:.*/root:x:18135:0:99999:7:::/g /etc/shadow\nRUN sed -i s/vco:.*/vco:x:18135:0:99999:7:::/g /etc/shadow" | docker build - -t vco_private:latest'
  3. Execute vracli cluster exec -- bash -c 'echo -e "FROM db-image_private:latest\nRUN sed -i s/root:.*/root:x:18135:0:99999:7:::/g /etc/shadow\nRUN sed -i s/postgres:.*/postgres:x:18135:0:99999:7:::/g /etc/shadow" | docker build - -t db-image_private:latest'
  4. Execute opt/scripts/backup_docker_images.sh to persist the new changes through reboots.

Scenario 2 : vRealize Automation 8.0/8.0.1 is already down as a result.

  1. SSH into each of the nodes
  2. Run opt/scripts/deploy.sh --onlyClean on a single vRA node to shutdown the services safely.
  3. Once completed, Repeat step 2 through 4 in Option 1 – > Scenario 1
  4. Run /opt/scripts/deploy.sh to start the services up.

Option 2

Apply the vRealize Automation 8.0.1 HF1 or HF2  with vRealize Lifecycle Manager 8.0.1 patch 1

Scenario 1 : vRealize Automation 8.0/8.0.1 is up and running

It is recommended to install vRealize Suite Lifecycle Manager 8.0.1 patch 1 before vRealize Automation 8.0.1 patch 1. The vRealize Suite Lifecycle Manager 8.0.1 Patch 1 contains a fix for some intermittent delays in submitting the patch request.

Apply vRealize Automation 8.0.1 patch 1 leveraging vRealize Lifecycle manager 8.0.1 Patch 1.

Scenario 2 : vRealize Automation 8.0/8.0.1 is already down as a result.

  1. SSH into each of the nodes
  2. Run /opt/scripts/deploy.sh --onlyClean on a single vRA node to shutdown the services safely.
  3. Once completed, Repeat step 2 through 4 in Option 1 – > Scenario 1
  4. Run /opt/scripts/deploy.sh to start the services back.Apply vRealize
  5. Apply Automation 8.0.1 patch 1 leveraging vRealize Lifecycle manager 8.0.1 Patch 1

Note: We highly recommend to be always on the more recent builds and patches.

New Deployments

If you need a video tutorial on how to install vRealize Automation 8.x check either my Youtube video on how to deploy vRA 8.x with vRealize Easy Installer here or my previous blog post here which also include the video.

Please subscribe and smash that tiny notification bill to get notified of any new and upcoming videos if you do check my Youtube channel.

Now that is out of the way , for new deployments of 8.0.1 and until 8.1 is released where the issue is resolved, it is really very simple.

Once you see that vRA 8.0.1 is deployed via vRealize suite lifecycle manager 8.0.1 and that its now reachable via the network, do the following:

  1. SSH into the vRA node
  2. Execute Kubectl get pods -n prelude to see if vRA started to deploy a few of the services in the prelude namespace.
  3. Once confirmed proceed to step 4
  4. Execute vracli cluster exec -- bash -c 'echo -e "FROM vco_private:latest\nRUN sed -i s/root:.*/root:x:18135:0:99999:7:::/g /etc/shadow\nRUN sed -i s/vco:.*/vco:x:18135:0:99999:7:::/g /etc/shadow" | docker build - -t vco_private:latest'
  5. Execute vracli cluster exec -- bash -c 'echo -e "FROM db-image_private:latest\nRUN sed -i s/root:.*/root:x:18135:0:99999:7:::/g /etc/shadow\nRUN sed -i s/postgres:.*/postgres:x:18135:0:99999:7:::/g /etc/shadow" | docker build - -t db-image_private:latest'
  6. Execute opt/scripts/backup_docker_images.sh to persist the new changes through reboots.
  7. Keep checking the status of the pods by continually running and executing Kubectl get pods -n preludeuntil all the pods are up and running.

If your only installing one appliance and you noticed that the vco-app pod status is CrashLoopBackOff 

2020-03-28_14-00-53

You will need to delete the pod so a new one gets provisioned from the newly updated docker build that we generated in step 4 by executing the following below command.

kubectl delete pods -n prelude vco-app-pod-name

If your installing a cluster and since we can’t simply delete the postgres pod to fix it –So the other postgres instances on the remaining nodes are able to replicate data-otherwise other services that depends on postgres will also fail so its better to just shutdown all the services on each of the nodes and doing the following:

  1. SSH into the vRA node
  2. Execute Kubectl get pods -n prelude to see if vRA started to deploy a few of the services in the prelude namespace.
  3. Execute /opt/scripts/deploy.sh --onlyClean on each of the nodes to stop the services.
  4. Once completed execute the workaround repeating step 4 through 6
  5. Run /opt/scripts/deploy.shon each of the nodes to start the services up.

Once your appliance or cluster is up and running apply the vRealize Automation 8.0.1 HF1 or HF2. ( Soon to be also released ) as I mentioned above in Option 2 for Existing Deployment.

If you have already one appliance with HF1 you can’t scale out to create a cluster since the original image is not patched with HF1. So unfortunately you have to wait a couple more weeks until 8.1 is out, where then you can upgrade then scale out your deployment to create a cluster production ready deployment.

If you do have any questions please post them below. I will try my best to have them answered.

Hope this has been hopeful if you have made it to the end.

The End Eh!

Automation and Orchestration vRealize Automation

vSphere Customization with Cloud-init While Using vRealize Automation 8 or Cloud.

After spending an enormous amount of time, which I think started somewhere in the summer of last year to get vSphere Customization to work with Cloud-init while using vRealize Automation 8 or vRealize Automation Cloud as the automation platform to provision virtual machine deployments and install, configure the applications running on it.

I finally have a workaround that I can say is guaranteed to work every single time, until something better comes along. i.e. Software components like we have it today within the vRA 7.x platform, which is planned to be released for vRA 8.x in Q3-2020 if everything goes as planned.

With some out-of-the-box thinking, I was able to use IP static assignment ( assignment: static ) within the vRA blueprints to leverage the IP Static pool and the network metadata that we define in vRA via Network Profiles for the targeted networks we want to connect to, while using cloud-init with Ubuntu 16.04 and Ubuntu 18.04 for now, but the principle should be the same for other Linux distributions, even though it seems that RHEL is the only OS today that just works provided traditional Guest OS Customization GOSC is set in cloud-init.

Note: The will also work if you were to use DHCP IP Assignment.

Hoping this was worth the time, I am documenting in this blog the step by step instructions on how to prepare your vSphere templates while leveraging cloud-init,  in addition to for your own reference, a list of all the internet available resources that I looked at while doing my research.

I will also have a video added to the blog later that showcases going through the entire template preparation and also demo after that a typical vRA 8 deployment using static IP assignment while leveraging cloud-init to install selected packages per machine component and execute various commands to setup an application.

I still say that this shouldn’t be that hard for our customers to setup and hopefully Software Component like I mentioned would save us all from all this complexity, of-course this is beside the fact that you still can do this via various configuration management tools such as Ansible and puppet which by the way vRealize Automation 8 and cloud integrate with today out-of-the-box.

How it works?

In a high level when the virtual machine first boots up and gets rebooted to be customized due to the dynamic vCenter customization specs that gets created based on the fact we are using the assignment static property ( assignment: static ) within the blueprint code as you see in the screenshot below, I am making sure that during that time, Cloud-init is in a disabled state.

2020-02-15_11-26-33

After the customization reboot the virtual machine once, there is a Cron Job that I created on the template that execute at startup after a 90 sec of sleep which is enough time for the virtual machine to be customized, rebooted and connected to the network without running the Cron Job as of yet. After the initial reboot and pass the 90sec mark now the Cron Job execute a shell script that enables cloud-init and initializes it running all the needed cloud-init modules. ( init, Config and Final)

Note: Feel free to increase the 90 sec if you feel you need more time as the virtual machine being customized. 

The End result, the virtual machine is now customized with an updated host-name and an IP from our targeted static IP pool configured for the network its connected to without having to hack the Cloud Config code any further to setup things like the host-name or even configure the network itself, and more importantly without conflicting with cloud-init which what the problem was all along.

Let’s get started, Eh!

Template Preparation Steps

  • Once the virtual machine is up and running update the list of available packages and install any new available version of these packages that you have to update your template.

sudo apt-get update && sudo apt-get -y upgrade

    • Install Cloud-init for Ubuntu 16.04. Ubuntu 18.04 have cloud-init pre-installed so you can skip this step

sudo apt-get -y install cloud-init

  • Configure OVF as your Datasource, then save and exit

sudo dpkg-reconfigure cloud-init

  • Enable traditional Guest OS Customization GOSC Script by editing /etc/cloud/cloud.cfg file and adding

disable_vmware_customization: true

  • Ensure network configuration is not disabled in /etc/cloud/cloud.cfg, by deleting or hashing the following if it exists:

network:
config: disabled

Or that any /etc/cloud/cloud.cfg.d/* configuration files, such as
99-disable-networking-config.cfg with the folloing content “network: {config: disabled}” doesn’t exist.

  • Set Temp not to clear, by editing /usr/lib/tmpfiles.d/tmp.conf  and adding the prefix # to line 11.

#D /tmp 1777 root root –

  • Configure Open-vm-tools to start after dbus.service by editing /lib/systemd/system/open-vmtools.service file and adding the following under the [Unit] section.

After=dbus.service

  • Reduce the raise network interface time to 1 min by editing /etc/systemd/system/network-online.targets.wants/networking.service file and changing: ( This not applicable on Ubuntu 18.04 )

TimeoutStartSec=5min to TimeoutStartSec=1min

  • Disable cloud-init on First Boot and until customization is complete by creating this file /etc/cloud/cloud-init.disabled

sudo touch /etc/cloud/cloud-init.disabled

  • Create a script your_script.sh in a known location that will be called by a Cron Job that will create later to enable and initialize cloud-init after the customization reboot. The script should contain the following commands:
sudo rm -rf /etc/cloud/cloud-init.disabled
sudo cloud-init init
sleep 20
sudo cloud-init modules --mode config
sleep 20
sudo cloud-init modules --mode final
  • Configure the script to be an executable

sudo chmod +x your_script.sh

  • Create a Cron Job that will run after 90 sec of sleep at boot by typing crontab -e and entering the following:

@reboot ( sleep 90 ; sh /Script_path/your_script.sh )

  • Copy the content below for the Template Cleaning script and create your_clean_script.sh. You can replace cloudadmin with your own user that you setup when you installed the Ubuntu OS
#!/bin/bash

# Add usernames to add to /etc/sudoers for passwordless sudo
users=("ubuntu" "cloudadmin")

for user in "${users[@]}"
do
cat /etc/sudoers | grep ^$user
RC=$?
if [ $RC != 0 ]; then
bash -c "echo \"$user ALL=(ALL) NOPASSWD:ALL\" >> /etc/sudoers"
fi
done

#grab Ubuntu Codename
codename="$(lsb_release -c | awk {'print $2}')"


#Stop services for cleanup
service rsyslog stop

#clear audit logs
if [ -f /var/log/audit/audit.log ]; then
cat /dev/null > /var/log/audit/audit.log
fi
if [ -f /var/log/wtmp ]; then
cat /dev/null > /var/log/wtmp
fi
if [ -f /var/log/lastlog ]; then
cat /dev/null > /var/log/lastlog
fi

#cleanup persistent udev rules
if [ -f /etc/udev/rules.d/70-persistent-net.rules ]; then
rm /etc/udev/rules.d/70-persistent-net.rules
fi

#cleanup /tmp directories
rm -rf /tmp/*
rm -rf /var/tmp/*

#cleanup current ssh keys
#rm -f /etc/ssh/ssh_host_*

#cat /dev/null > /etc/hostname

#cleanup apt
apt-get clean

#Clean Machine ID

truncate -s 0 /etc/machine-id
rm /var/lib/dbus/machine-id
ln -s /etc/machine-id /var/lib/dbus/machine-id

#Clean Cloud-init
cloud-init clean --logs --seed

#cleanup shell history
history -w
history -c

  • Configure the Template Cleaning script to be an executable as well

sudo chmod +x your_clean_script.sh

  • Make sure you can switch to user root by editing the fine /etc/ssh/sshd_config and changing PermitRootLogin to yes

PermitRootLogin yes

  • Set a password for root

sudo passwd root

Note: The reason for the above is to be able to execute the clean script with no issues as I personally had issues executing the clean up script with sudo working with the Ubuntu 18.04. you can always revert it back once the cleanup template is executed. 

  • Execute the Template Cleaning Script.

sudo ./Script_path/your_clean_script.sh

  • Shutdown the virtual machine and turn it into a template.

Shutdown -h now

Note : Just be aware that the cron job will also run if you try to update the template for any reason . So make sure if you do pass 90 sec while doing your change is to re-add the /etc/cloud/cloud-init.disabled file and then re-execute the clean up script again before shutting down the template .

if you dont cloud-init will execute on first boot and you will get the vm customization but your cloud config code wont be applied

Click To See It All In Action On my Youtube Channel !

 

Happy Template Building! Please share! 

The End Eh!

Resources:

https://ubuntu.com/engage/cloud-init-whitepaper
https://debconf17.debconf.org/talks/164/
https://cloudinit.readthedocs.io/en/latest/
https://events.linuxfoundation.org/wp-content/uploads/2017/12/Cloud-init-The-cross-cloud-magic-sauce_Smith_moser.pdf
https://www.youtube.com/watch?v=RHVhIWifVqU
https://www.youtube.com/watch?v=y8WA1BUlT-Q
https://linuxtechlab.com/executing-commands-scripts-at-reboot/
https://blogs.vmware.com/management/2019/02/building-a-cas-ready-ubuntu-template-for-vsphere.html
http://kb.vmware.com/s/article/56409
https://kb.vmware.com/s/article/59687
http://kb.vmware.com/s/article/59557
http://kb.vmware.com/s/article/2378666
https://blah.cloud/infrastructure/using-cloud-init-for-vm-templating-on-vsphere/
http://ubuntu.com/blog/cloud-init-v-18-2-cli-subcommands
http://lucd.info/2019/12/06/cloud-init-part-1-the-basics/

 

Blueprinting Cloud Automation Services Cloud-init vRA Blueprints vRealize Automation vSphere Customization

Part 3: vRealize Automation 8.0 Deployment with vRealize Suite Lifecycle Manager 8.0

In Part 2 of my vRealize Automation 8.0 blog video series, we have upgraded vRealize Lifecycle Manager 2.1 to 8.0 by performing a side by side migration leveraging the vRealize Easy Installer while importing the management of both VMware Identity manager 3.3.0 and the vRealize Suite 2018 environment.

In this blog video we will be using vRealize Lifecycle Manager 8.0 to deploy vRealize Automation 8.0 in a new environment.

Now as for requirements you will need :

  1. vRealize Lifecycle Manager 8.0
  2. VMware Identity Manager 3.3.1
  3. A new Hostname, IP Address and a DNS record for the new vRA 8.0 appliance that the vRealize Suite Lifecycle Manager 8.0 will be creating.
  4. Product Mapping is set with the install and upgrade binaries for the new vRealize Suite 2019 Products.

 

Deployment Workflow

2019-10-23_10-13-17

Please note that the installation process in the video after hitting submit is fast forwarded.

The End, Eh!

Automation and Orchestration vRealize Automation vRealize Suite vRealize Suite Lifecycle Manager

Part 2: Migration of vRSLCM 2.x Version to vRealize Suite Lifecycle Manager 8.0

If you happen to have an existing vRSLCM 2.x and vIDM 3.3.0 in your environment then you will need the vRealize Easy Installer to migrate your existing vRSLCM 2.x instance to vRSLCM 8.0.

Once your migration to vRSLCM 8.0 is completed you can upgrade your vIDM instance to 3.3.1 since its a requirement before you can install vRealize Automation 8.0 with vRealize Lifecycle Manager 8.0

Again as a reminder vRealize Automation 8.0 is installed, configured, managed and upgraded only through vRealize Suite Lifecycle Manager 8.0

Now as for requirements you will need :

  1. A new Hostname, IP Address and a DNS record for the new vRSLCM 8.0 appliance that the vRealize Easy Installer will be creating.
  2. To make sure that the password for the sshuser on the existing vIDM appliance is not expired.
  3. To enable root access for SSH on the existing vIDM appliance following VMware KB 2047626
  4. To Download the install and upgrade binaries for vRealize Suite 2019
  5. To Make sure you have enough storage on the new vRSLCM 8.0 appliance.

Migration Workflow

migration flow

Please note that the installation process in the video after hitting submit is fast forwarded.

NOTE (vIDM Upgrade Support )  :

  • Green-field vRealize Suite Lifecycle Manager 8.0 supports only 3.3.1 version of VMware Identity Manager to be installed or imported.
  • Older versions (2.9.2, 3.2.0, 3.2.0.1 & 3.3.0) of VMware Identity Manager will be supported only for existing vRealize Suite Lifecycle Manager instances that are being migrated to vRealize Suite Lifecycle Manager 8.0.
  • Upgrade support from older VMware Identity Manger to latest is only available if they conform to vRealize Suite Lifecycle Manager supported form-factor.
  • Versions prior to vRealize Suite Lifecycle Manager 8.0 allowed only single instance of VMware Identity Manager to be deployed with embedded connector and embedded postgresql database.
  • Upgrade for VMware Identity Manager within vRealize Suite Lifecycle Manager 8.0 to latest versions will only be supported if it conforms to the above mentioned form-factor.

Else the upgrade has to be performed outside vRealize Suite Lifecycle Manager and Once upgraded, it can any-time be re-imported by triggering Inventory Sync in vRealize Suite Lifecycle Manager 8.0

 

The End, Eh!

Helpful Links You Might Need

Resetting the admin@localhost password in vRealize Suite Lifecycle Manager

Restting root password on photon OS

vRealize Automation vRealize Suite Lifecycle Manager

Part 1: vRealize Automation 8.0 Simple Deployment with vRealize Easy Installer

On October 17th, 2019 VMware announced the next major release of vRealize Automation. it uses a modern Kubernetes based micro-services architecture and brings vRA cloud capabilities to the on-premises form factor.

What’s New

The many benefits of vRA 8.0 include:

  • Modern Platform using Kubernetes based micro-services architecture that provides
  • Easy to setup and consume multi-cloud infrastructure surface
  • Embedded vRO 8.0 Web Client and Orchestrator’s new release features
  • Deliver Infrastructure-as-Code using a declarative YAML syntax
  • Cloud Agnostic Blueprints
  • Iterative development of Blueprints
  • Self-service catalog coupled with agile governance
  • Collaboration across teams via sharing of objects
  • Kubernetes/container management
  • Deploy IPv6 workloads on dual-stack IP (IPv4/ IPv6) networks in vSphere
  • CI/CD pipeline and automated application release management
  • New Action based extensibility (ABX), which allows you to write lightweight scripts, using node.js and python.
  • Git Integration to manage all blueprints, workflows, actions and pipelines.

For more information, kindly refer to the Release Notes

vRealize Automation 8.0 is installed, configured, managed and upgraded only through vRealize Suite Lifecycle Manager 8.0 .

In the video posted below, I’am going to provide the step-by-step process of using the vRealize Easy Installer to :

  • Install vRealize Suite Lifecycle Manager 8.0
  • Deploy VMware Identity Manager 3.3.1 and register with vRealize Automation.
  • Install new instance of vRealize Automation 8.0

 

Installation Workflow

installer workflow

Please note that the installation process in the video after hitting submit is fast forwarded.

The End, Eh!

Automation and Orchestration vRealize Automation

vRealize Automation 7.6 (vRA 7.6) ITSM 7.6 Plug-in for ServiceNow

VMware vRealize Automation is a hybrid cloud automation platform that transforms IT service delivery. With vRealize Automation, customers can increase agility, productivity and efficiency through automation, by reducing the complexity of their IT environment, streamlining IT processes and delivering a DevOps-ready automation platform.

If you want to know more about the upcoming release of vRealize Automation 8, please check out our updated product page here.

In this blog we will be focusing on installing and configuring the new ITSM 7.6 Plug-in which was released and currently only available on the Service Now Store here for vRealize Automation 7.6, 7.5 and 7.4.

Summary

The vRealize Automation plugin 7.6 for ServiceNow provides an out of the box integration between the ServiceNow portal and vRealize Automation catalog and governance model. It enables ServiceNow users to deploy virtual machines and perform basic day 2 operations on their CMDB assets.

4

Once you install and configure the plug-in, vRealize Automation catalog items which are entitled to ServiceNow users will automatically appear in a special ServiceNow vRealize Automation portal.

When leveraging vRealize Automation’s ecosystem items via the ServiceNow portal, the vRealize Automation ServiceNow plugin will allow you to directly benefit from the extensibility and governance capabilities of vRealize Automation.
You can additionally leverage all vRealize Automation Event Broker integrations and include vRealize Automation approval policies.

Key Features

  • The plugin enables integration between vRealize Automation with ServiceNow platform to provide the ability for ServiceNow users to access the vRealize Automation catalogs & resources within ServiceNow.
  • The plugin allows ServiceNow users to request vRealize Automation catalog items from ServiceNow portal.
  • The plugin extends ServiceNow functionality to be able to render vRealize Automation catalog items into ServiceNow dynamically and manage vRealize Automation resources.
  • Day2 operations performed on the resources in ServiceNow CMDB will be synced back to vRealize Automation.
  • The Plugin can support multiple vRealize Automation Instances within the same ServiceNow Instance

What’s New

  • Resource Sharing and entitlements across ServiceNow users
  • Resource visibility based on Entitlements within ServiceNow
  • Header Rebranding where Global admin/plugin admin can apply changes based on his requirements.
  • Footer Rebranding where Global admin/plugin admin can apply changes to images, logo, text, colors, based on his requirements.
  • Two way checkout functionality for Catalog items
  • Boolean Yes_No field type support
  • Date/Time field support
  • Hyperlink field type support
  • Reconcile of CMDB
  • Business group functionality
  • Support for cross reference / Business group properties
  • Custom property fields population based on Business group selection

5

Updates and Improvements

  • Enhanced Documentation which you have to refer to.
  • ITSM 7.6 supports London and Madrid version of ServiceNow.
  • ITSM 7.6 support vRealize Automation 7.6, 7.5 and 7.4.
  • UI Improvement for Category widget on Service Portal.
  • UI improvement for browser Scrollbar on request and catalog item page on portal.
  • Issue with Catalog client script (Regex Function support).

A few facts first!

Update ITSM Application for ServiceNow

If you have previously downloaded the vRealize Automation ITSM Application version 5.1 from the ServiceNow store portal, you can update it to version 7.6 in your instance using the same portal. More details around the actual update procedure can be found in the documentation.

If you have a deployment of 5.1 which was downloaded and installed from VMware Marketplace or any of the following :

  • ITSM v5.1 Downloaded from VMware Marketplace
  • ITSM v5.0 Downloaded from VMware Marketplace
  • ITSM v4.1 Downloaded from VMware Marketplace
  • ITSM v4.0 Downloaded from VMware Marketplace
  • ITSM v3.0 Downloaded from VMware Marketplace

Before your deployment can be updated, the current version of the vRealize Automation ITSM Application must be uninstalled with the help of ServiceNow. Please open a ticket with ServiceNow at https://hi.service-now.com/hisp to remove the application ‘VMware vRealize Automation ITSM Application’.

Install a MID Server

  • Use or Install a Management, Instrumentation, and Discovery (MID) Server to facilitate the communication between ServiceNow and vRealize Automation.
    • Check my Pervious Blog here, on how to do that Or
    • Check Video 2 below

Install ITSM Application for ServiceNow

I usually would have captured the entire process but unfortunately I don’t have access to the ServiceNow Store portal so I would have to install the plug-in similar to how I did in my pervious blog for the 5.0/5.1 Plug-in here.

Now assuming you have access, let’s follow the following steps :

1

  • Click on the application to view the details. On the top right corner of the form, you will see buttons to “Purchase” and “Manage Entitlements”
  • The vRealize Automation ITSM Application for ServiceNow is free. 
  • Click “Manage Entitlements” and select the ServiceNow instances where the application should be installed. Click OK.
  • The application should now be available on the ServiceNow instances selected in the previous step.
  • Log in to the ServiceNow Instance as a ServiceNow system administrator.
  • Select System Applications > All Available Applications > All. 
  • Search for the application “vRealize Automation ITSM Application for ServiceNow
  • Click Install. In the popup, please select Install with demo data and complete the installation

This completes the installation of the Application.

Configure ITSM Plug-in for ServiceNow

After installation, you need to carry out the following configuration steps. 

Enable Application Access on Tables

  • You must enable application access to certain tables for the application to work.
    • Check my Pervious Blog here, on how to do that in Step 1 Or
    • Check Video 4 below.

Set up ServiceNow Users for managing the vRealize Automation ITSM Plug-in

Setup the ServiceNow users who will manage the vRealize Automation ITSM application configuration and enable end users to use the vRealize Automation User Portal.

The table below captured from the documentation describes the persona and necessary roles to enable the persona. 

2

  • vRealize Automation and ServiceNow may be backed by different Authentication Providers. It is important to setup the users in both systems with the same email address. This email address is used to match the user records across the Authentication Providers of the respective systems. The correlation is required to correctly assign the ownership of the deployments and machines. 
  • Authentication, Roles and Entitlements are defined and enforced by ServiceNow. They have no correspondence in vRealize Automation. 

Set up ServiceNow Users for approval and support the vRealize Automation ITSM Plug-in

Setup the ServiceNow users who will approve the requests for deployments. Also, setup the users who will receive a support ticket on request failures. 

The table below captured from the documentation describes the persona and the necessary groups to enable the approval and support notifications. 

3

Note: The Support group is actually called “vRA ServiceNow Support Group” and not “vRealizeAutomation-Support

Set up the integration user in vRealize Automation

You must set up a user in the vRealize Automation. The vRealize Automation ITSM Application connects to vRealize Automation using the credentials of this user to perform all actions including import of catalog items and categories, deployments and its resources, and requests for new deployments. 

The integration user must be a business group manager within the business groups that you want ServiceNow to manage. The integration user does not require a role within ServiceNow.

  • Check my Pervious Blog here, on how to do that in Step 5 Or
  • Check Video 6 below.

All Catalog Item Requests from ServiceNow are serviced by this integration user in vRealize Automation. In vRealize Automation, the requests and corresponding deployments will show the integration user as the owner. However, in ServiceNow, the requests and corresponding deployments will show the ServiceNow user who initiated the request. This is achieved by correlating the Email address from the user records across two systems.

Now that we got that out of the way lets dive right in with this series of videos that will capture the integration workflow.

6

Video 1: Requesting a ServiceNow Developer Instance

Video 2: Setting Up The ServiceNow MID Server

Video 3: Setup The Plug-in To Be Accessible From All Application Scopes

Video 4: Enable Plug-in Access To Certain Tables

Video 5: Installing The ITSM 7.6 Plug-in

Video 6: Setting Up The Integration User

Video 7: Setup ServiceNow Users (Admin, User, Approver and Support)

Video 8: Register vRealize Automation Instance In ServiceNow

Video 9: Creating vRealize Automation Entitlements

 

Coming Soon!

Video 10: Request vRealize Automation Catalog Item In ServiceNow

Video 11: Sharing Resources in ServiceNow vRealize Automation User Portal

Video 12: Rebrand the vRealize Automation User Portal in ServiceNow

 

Uncategorized

VMware Cloud Automation Services (CAS) – Cloud Assembly – Part 1

VMware’s cloud automation services are a set of cloud services that leverage the award-winning vRealize Automation on-premises offering. These services make it easy and efficient for developers to build and deploy applications. The cloud automation services consist of VMware Cloud Assembly, VMware Service Broker, and VMware Code Stream. Together, these services streamline application delivery, enable cloud flexibility and choice, and control risks. Additionally, these services facilitate collaboration between traditionally siloed groups helping further with accelerating business innovation.

  • VMware Cloud Assembly: Developers want the same experience of automating deployment and consumption of infrastructure and applications in private and hybrid clouds as they adopt public clouds. Cloud Assembly delivers unified provisioning across all clouds through declarative Infrastructure as Code, including AWS, Azure, and VMware Cloud on AWS. With Cloud Assembly, IT and cloud operations teams can orchestrate and expedite infrastructure and application delivery in line with DevOps principles, improving the overall developer experience, developers get an experience equivalent to provisioning resources from native public clouds.
  • VMware Service Broker: Service Broker provides simple, self-service access to multi-cloud infrastructure and application resources from a single catalog, without requiring disparate tools. With Service Broker, operations teams can more effectively govern resource access, and use and enforce security, deployment and business policies across multi-cloud environments.
  • VMware Code Stream: Enterprise development teams are creating and iterating on applications faster than ever, but this work is often delivered using a combination of manual scripting and a mix of delivery tools. This creates challenges with delivery speed, visibility, and troubleshooting for code releases. Code Stream automates the code and application release process with a comprehensive set of capabilities for application deployment, testing, and troubleshooting. It features integrations with popular developer tools and supports VMware-based private clouds, VMware Cloud on AWS and native public clouds. With Code Stream enterprises get code and applications out faster and reduce the time it takes to correct issues when they arise.

In part 1 of 2 of this blog post we will explore CAS and how to initially set it up and configure it starting with a new assigned Cloud Organization registered with the above mentioned Cloud Automation Services.

You can access VMware Cloud Services by visiting https://cloud.vmware.com/  then clicking on the Log In menu option to use the services.

cas-01

Once you sign in with your credentials you will have access to the Console Menu option which will take you in to access the Cloud Automation Services we mentioned above.

cas-02

cas-03

Most of the work we will be doing will be initially in VMware Cloud Assembly then we will be able to extend the work to the other two services, VMware Service Broker and the VMware Code Stream respectively.

In this setup we will be leveraging the following environments :

  • VMware SDDC Cloud ( Home Lab ) as my Production Environment.
  • AWS EC2 Cloud as my Development Environment.
  • Azure Cloud as my Testing Environment.

So let’s get started, Eh!

VMware Cloud Assembly

VMware Cloud Assembly is an infrastructure as code automation solution designed to expedite infrastructure consumption and application delivery in line with DevOps principles, through an intuitive symmetrical dual interface ( Code or Draw ) that supports declarative, intent-based application infrastructure provisioning, blueprint design and lifecycle management across multiple clouds such as VMware SDDC, Amazon Web Services, VMware Cloud on AWS and Azure as a minimum on GA date.

The infrastructure as code approach streamlines infrastructure consumption by enabling blueprint parameterization, iteration on blueprint development and easy version control through native controls or supported version control systems. The SaaS form factor enables VMware to keep the platform up to date, allowing operations teams to focus on higher value activities, such as business systems reliability and performance.

Our intent here to use Cloud Assembly to provision new projects to public clouds that we can then bring on-prem when they are ready to go to production for example. My goal here really is to show you how we can create an agnostic blueprint ( WordPress Application ) that the user can deploy from Cloud Assembly Directly or request it from the Service Broker ( Catalog ) and selecting which environment ( Dev, Test, Prod ) you want to deploy the application to.

And to do that we need to configure a few things to create our deployment stack and start deploying blueprints

  1. Create our Cloud Accounts.
  2. Create Cloud Zones.
  3. Create one or more Projects.
  4. Create Flavor mapping.
  5. Create Image mapping
  6. Create and deploy blueprints.

Cloud Accounts

Cloud accounts allow you to bring your public cloud and on-prem data centers under CAS management.

In Cloud Assembly, navigate to  Infrastructure > Connections > Cloud Accounts > ADD CLOUD ACCOUNT

cas-04

We will configure our 3 Account Types / Environments here :

cas-05

vCenter Account Type

Prerequisites Checklist

  1. You have at least one collector / Cloud Proxy VM installed.
  2. You have the vCenter IP address/FQDN.
  3. You have the vCenter user name and password.

Note that a collector VM can typically support 10,000 VMs

Installing Cloud Proxy

  • Click Add Cloud Account
  • Select vCenter as the account type. When you don’t have any previous Cloud Proxies setup, you will be presented with the steps needed to install one.

cas-06

  • Download the Cloud Proxy ova file to deploy it in vCenter OR
  • Copy the OVA link to directly deploy it in vCenter without having to download it first.
  • Import the .ova file to the vCenter Server and start the installation following the standard steps provided by the OVF Deployment Wizard.

cas-07

  • Once you get to the Customize Template section within the OVF Deployment Wizard we will provide the following properties:
    • CAS One Time Key (OTK)
    • Root User name and password
    • Remote Data Collector / Cloud Proxy Display Name in CAS
    • Network Proxy Settings ( Optional )
    • Networking Properties

cas-08

  • Click Next and Finish to deploy the cloud proxy. it takes a few minutes to detect your Cloud Proxy after it is deployed and powered up in vCenter.
  • To verify the detection of the Cloud Proxy is complete navigate to Infrastructure > Connections > Cloud Proxies and verify its listed with a good status.

cas-09

  • Navigate again to Infrastructure > Connections > Cloud Accounts > ADD CLOUD ACCOUNT 
  • Click Add Cloud Account
  • Select vCenter as the account type. Now that we setup a Cloud Proxy, you will be able fill all the requirement including the Cloud Proxy we just deployed.
  • Enter the vCenter user name and password and hit VALIDATE. 
  • Provide a Name and a description for the Cloud Account.
  • In the Configuration Section select which DataCenters you want to allow provisioning to
  • Allow to create a Cloud Zone for the Selected Datacenters by checking the check box, this will automatically create the Cloud Zone for us so we don’t have to later.
  • Add Capability Tags as required, this will be used for placement decisions as we will see later in the blog post.
  • Click ADD when Completed.

cas-10

  • Once added you should see the cloud account listed with OK Status

cas-11

Amazon Web Services Account Type

Prerequisites Checklist

  1. Access Key ID
  2. Secret Access Key
  • Navigate again to Infrastructure > Connections > Cloud Accounts > ADD CLOUD ACCOUNT 
  • Click Add Cloud Account
  • Select AWS Web Services as the account type.
  • Provide the Access Key ID and Secret Access Key and Click VALIDATE
  • Provide a Name and a description for the Cloud Account.
  • In the Configuration Section select which Regions you want to allow provisioning to.
  • Allow to create a Cloud Zone for the Selected Region by checking the check box, this will automatically create the Cloud Zone for us so we don’t have to later.
  • Add Capability Tags as required, this will be used for placement decisions as we will see later in the blog post.
  • Click ADD when Completed.

cas-12

  • Once added you should see the cloud account listed with OK Status

cas-13

Azure Account Type

Prerequisites Checklist

  1. Subscription ID
  2. Tenant ID
  3. Client Application ID
  4. Client Application Secret Key

Note: If you want to know how get these IDs, this is very similar to how we currently setup up vRealize Automation Azure endpoint and there are plenty of blogs you can reference such as my personal favourite by Jon Schulman

  • Navigate again to Infrastructure > Connections > Cloud Accounts > ADD CLOUD ACCOUNT 
  • Click Add Cloud Account
  • Select Azure as the account type.
  • Provide the IDs required and Click VALIDATE
  • Provide a Name and a description for the Cloud Account.
  • In the Configuration Section select which Regions you want to allow provisioning to.
  • Allow to create a Cloud Zone for the Selected Region by checking the check box, this will automatically create the Cloud Zone for us so we don’t have to later.
  • Add Capability Tags as required, this will be used for placement decisions as we will see later in the blog post.
  • Click ADD when Completed.

cas-14

  • Once added you should see the cloud account listed with OK Status

cas-15

Cloud Zones

Cloud zones associate compute resources with projects and account/regions to form the basis of deployable virtual machines. In addition, they enable you to define capabilities that Cloud Assembly matches with blueprint constraints to define where and how resources are configured for deployments.

Now remember that we checked the check box to create a cloud zone for the selected Datacenter/Region where we want to provision to for each of the cloud account types we have created. (vSphere, AWS and Azure )

Navigate to Infrastructure > Configure > Cloud Zones to list the pre-created Cloud Zones or to create new ones if you decide for example to add new Datacetners / Regions to provision machines to.

You don’t need to create any cloud zones if you selected the option to automatically create zones when you added your cloud accounts, we will customize the cloud zones in this section of the blog.

cas-16

Within each of the cloud zone > Summary Tab you can select a Placement Policy that defines how provisioned resources are distributed among hosts in this cloud zone. By default resources are placed on random hosts but:

One of the following strategies can be optionally applied:

  • BINPACK – Will place computes on the most loaded host that still has enough resources to run the given compute.
  • SPREAD – Will attempt to spread computes evenly across hosts.

For the purpose of the blog we will leave the Placement Policy as DEFAULT for all the Cloud Zones.

cas-17

You saw me mention the use of Capability tags and so far we have created tags on the Cloud Account type level.

cas-18

When it comes to Tagging Strategy you must carefully plan and implement an appropriate tagging strategy based on your organization’s IT structure and goals to maximize Cloud Assembly functionality and minimize potential confusion.

Tags are a critical component of Cloud Assembly that drive the placement of deployments through matching of capabilities and constraints. You must understand and implement tags effectively to make optimal use of Cloud Assembly. you also need to create an outline of your strategy and make it available to all users with privileges to create or edit tags.

For best practices for Tagging and Tagging Implementation, I would recommend spending few minutes first reading the documentation on What Are Tags.

vSphere Cloud Zone

Like we mentioned already a cloud zone defines a set of compute resources that can be used for provisioning.  In our Toronto Datacenter we have two clusters TOR-COMP-CL and TOR-MGMT-CL out of which we only want to use the TOR-COMP-CL for provisioning.

cas-19

Within the vSphere Cloud Zone and under the Compute tab we have what we call Filter Tags which we will use to remove or filter out the TOR-MGMT-CL from the cloud zone compute resource list since its our management cluster and it will not participate in being used as a resource we can provision workloads to.

At the beginning of the blog we mentioned that will be using our vSphere environment for our Production workloads so will we need to add a capability tag that we can use later in our blueprints as a constrain if we want to target our compute production cluster.

To do that we will first select the TOR-COMP-CL > Click TAGS, Type Env:Production and hit enter to form the tag then click Save

cas-20

Now we can also use the Env:Production tag to filter out the compute resource TOR-MGMT-CL by using the tag as a Filter Tag and list only those compute clusters that has the same tag, also like I mentioned I don’t want the TOR-MGMT-CL cluster to be part of the vSphere Cloud Zone at all.

cas-21

Now you may ask but why would you want to do that if you can simply use the Env:Production tag to target the TOR-COMP-CL cluster . What a great question I might say?

Its all about planning, where if for example I added another production cluster within the same Datacenter in the future, I can then simply tag it with the same tag  then leverage a higher level tag like the one we setup on the vCenter Account Type which we also could have setup on the Cloud Zone level within the Summary Tab to target all the vSphere production clusters. That cloud account tag was Cloud:vSphere which will allow me to target all my production clusters

Again you really don’t have to do that as I m just trying to prove a point here, as this can be done in many different ways.

AWS Cloud Zone

When we added the AWS cloud Account we selected the CA-Central-1 Region as the region we want to provision our development workload to. As you can see in the Screen shot below the AWS cloud Zone has two compute resources / availability zones that I can target for my Development workloads.

cas-22

Since I am okay utilizing all the AWS compute resources listed within the cloud zone I can place my Env:Development capability tab on the cloud zone level within the Summary Tab instead of placing it on the compute level like we did previously with the vSphere cloud zone. Click SAVE when your done.

cas-23

Azure Cloud Zone

For Azure when we created the Azure Account we selected the EAST US as the region we want to provision our test workload to. As you can see in the Screen shot below the Azure cloud Zone has one compute resources / availability zones that I can target for my Testing workloads.

cas-24

Just like the AWS Cloud Zone I can place my Env:Testing capability tag on the cloud zone level within the Summary Tab. Click SAVE when your done.

cas-25

So in summary we have added 3 different cloud account types, selected the Datacenter / Regions we want to provision workloads to and created there respective Cloud Zones and added capability tags on the Cloud Account Level , Cloud Zone level and Cloud Zone Compute level as we see fit, that we can leverage later as constraints when we create/design our blueprints.

Flavor Mapping

Cloud vendors use flavors, or instance types, to express standard deployment sizings such as small or large for compute resources. when we create a blueprint, you need to pick a flavor.

Flavor mappings are of course regional settings. This becomes critical in public cloud endpoints where sizes are dictated by a phrase like T2.Micro in AWS as opposed to fixed sizing details like in vCenter that might equate to a specific number of CPUs or GBs of Memory.

We will define three flavor mapping ( Small, Medium, Large ) across vSphere, AWS and Azure.

Small Flavor Mapping

  • In Cloud Assembly, navigate to Infrastructure > Configure > Flavor Mappings
  • Click + NEW FLAVOR MAPPING
  • Enter Flavor Name : Small
  • Click on Search for regions and create a Small Flavor Mapping for all 3 Clouds
  • Click CREATE

cas-26

Medium Flavor Mapping

  • In Cloud Assembly, navigate to Infrastructure > Configure > Flavor Mappings
  • Click + NEW FLAVOR MAPPING
  • Enter Flavor Name : Medium
  • Click on Search for regions and create a Medium Flavor Mapping for all 3 Clouds
  • Click CREATE

cas-27

Large Flavor Mapping

  • In Cloud Assembly, navigate to Infrastructure > Configure > Flavor Mappings
  • Click + NEW FLAVOR MAPPING
  • Enter Flavor Name : Large
  • Click on Search for regions and create a Large Flavor Mapping for all 3 Clouds
  • Click CREATE

cas-28

Once completed you should have 3 Flavor Mappings ( Small, Medium, Large ) for the 3 cloud platforms ( vSphere, AWS, Azure )

cas-29

Image Mapping

Cloud vendors use images to configure a VM based on OS Settings, such as an ubuntu-16 configuration. When you build a blueprint, you pick an image that fits your needs.

An image mapping associates a defined image name with a machine template. You can create one or more image names and map to a metadata file that contain pre-defined value sets. For example, an image might map to an OVA file that contains pre-populated cost or region specifications to import into the blueprint. Image mappings again are regional settings.

we will define three image mapping ( CentOS7, Ubuntu, Windows 2016 ) across vSphere, AWS and Azure.

Windows 2016 Image Mapping

  • In Cloud Assembly, navigate to Infrastructure > Configure > Image Mappings
  • Click + NEW IMAGE MAPPING
  • Enter Flavor Name : Windows 2016
  • Click on Search for regions and for images to create a Windows 2016 Image Mapping for all 3 Clouds
  • Click CREATE

cas-30

CentOS7 Image Mapping

  • In Cloud Assembly, navigate to Infrastructure > Configure > Image Mappings
  • Click + NEW IMAGE MAPPING
  • Enter Flavor Name : CentOS7
  • Click on Search for regions and for images to create a CentOS7 Image Mapping for all 3 Clouds
  • Click CREATE

cas-31

Ubuntu Image Mapping

  • In Cloud Assembly, navigate to Infrastructure > Configure > Image Mappings
  • Click + NEW IMAGE MAPPING
  • Enter Flavor Name : Ubuntu
  • Click on Search for regions and for images to create a Ububtu Image Mapping for all 3 Clouds
  • Click CREATE

cas-32

Once completed you should have 3 Image Mappings ( Ubuntu, CentOS7, Windows 2016 ) for the 3 cloud platforms ( vSphere, AWS, Azure ).

cas-33

Network Profiles

A network profile defines a group of networks and network settings that are available for a cloud account in a particular region or datacenter. A network profile defines the networking options and capabilities that are made available to deployed machines, based on the network tags in the network component YAML in a blueprint.

You typically define network profiles to support a target deployment environment, for example a small test environment where an existing network has outbound access only or a large load-balanced production environment that needs a set of security policies. Think of a network profile as a collection of workload-specific network characteristics.

AWS Network Profile

  • In Cloud Assembly, navigate to Infrastructure > Configure > Network Profiles
  • Click + NEW NETWORK PROFILE
  • In the Summary tab, For Account / Region click Search for regions and select your AWS Region.
  • Enter a Network Profile Name : AWS Network Profile
  • Enter a Capability Tag : Env:Development  This is again because we are using AWS as our development environment.

cas-34

  • In the Networks Tab, Select + ADD NETWORK  and select the discovered network or networks to use when provisioning a VM.
  • Select the added Network and assign a capability tag, for example here I setup Type:BackEnd-net and Type:FrontEnd-net as tags where both networks support Public IPs.
  • In the Security tab, I have added the default Security group that enables RDP and SSH Inbound and all communications Outbound within the selected VPC in the CA-Central-1a and CA-Central-1b zones.
  • Click CREATE

cas-35

Azure Network Profile

  • In Cloud Assembly, navigate to Infrastructure > Configure > Network Profiles
  • Click + NEW NETWORK PROFILE
  • In the Summary tab, For Account / Region click Search for regions and select your Azure Region.
  • Enter a Network Profile Name : Azure Network Profile
  • Enter a Capability Tag : Env:Testing  This is again because we are using Azure as our Testing environment.

cas-36

  • In the Networks Tab, Select + ADD NETWORK  and select the discovered network or networks to use when provisioning a VM.
  • Select the added Networks and assign a capability tag, for example here I setup Type:BackEnd-net and Type:FrontEnd-net as tags where both networks support Public IPs.
  • In the Security tab, I have added the vmwarelabnetworksecurity Security group that enables RDP and SSH Inbound and all communications Outbound within the selected Network Domain in the East US Zone.
  • Click CREATE

cas-37

vSphere Network Profile

  • In Cloud Assembly, navigate to Infrastructure > Configure > Network Profiles
  • Click + NEW NETWORK PROFILE
  • In the Summary tab, For Account / Region click Search for regions and select your vSphere Datacenter.
  • Enter a Network Profile Name : vSphere Network Profile
  • Enter a Capability Tag : Env:Production This is again because we are using vSphere as our Production environment.

cas-38

  • In the Networks Tab, Select + ADD NETWORK  and select the discovered network or networks to use when provisioning a VM.
  • Select the added Networks and assign a capability tag, for example here I setup Type:FrontEnd-net as a tag where the network support Public IPs. Since this is vSphere, this means that the  network can access internet and not necessarily have an actual public IP like in AWS or Azure.
  • The reason there is no Security tab in Network Profile for vSphere at this point is because we have not setup any NSX account types associated with the vCenter cloud account type.
  • Select the network and click on MANAGE IP RANGES > + NEW IP RANGE to define the set of IP addresses that can be reserved during provisioning.

cas-41

cas-42

  • Click CREATE

cas-40

Once completed you should have 3 Network Profiles for the 3 cloud platforms ( vSphere, AWS, Azure ).

cas-43

Storage Profiles

A storage profile is a cloud-specific set of storage policies that let the cloud administrator define storage for a cloud account region. Storage polices include disk customization, and a means to identifying the type of storage by applying capability tags. Tags are then matched against blueprint constraints to create the desired storage at provisioning time.

AWS Storage Profile

EBS Fast Storage

  • In Cloud Assembly, navigate to Infrastructure > Configure > Storage Profiles
  • Click + NEW NETWORK PROFILE
  • For Account / Region click Search for regions and select your AWS Region.
  • Enter a Storage Profile Name : AWS Storage Profile EBS Fast
  • Enter Device Type : EBS
  • Enter Volume Type : Provisioned IOPS SSD (IO1)
  • Enter Max IOPS : 800 and Select Support Encryption
  • Enter a Capability Tag : Env:Development  and Gold This is again because we are using AWS as our development environment and leveraging the fastest Disks

cas-44

Storage profiles are being grouped by Cloud Accounts. Now that we created the first storage profile for AWS, we will add additional profiles for the storage types we want to support such as a Slow Storage.

EBS Slow Storage

  • In Cloud Assembly, navigate to Infrastructure > Configure > Storage Profiles
  • Click + NEW NETWORK PROFILE
  • For Account / Region click Search for regions and select your AWS Region.
  • Enter a Storage Profile Name : AWS Storage Profile EBS Slow
  • Enter Device Type : EBS
  • Enter Volume Type : General Purpose SSD (GP2)
  • Select Support Encryption
  • Enter a Capability Tag : Env:Development  and Silver This is again because we are using AWS as our development environment and leveraging the Slower Disks

cas-45

Once completed will end up with two EBS Storage Profiles for AWS

cas-46

vSphere Storage Profile

  • In Cloud Assembly, navigate to Infrastructure > Configure > Storage Profiles
  • Click + NEW NETWORK PROFILE
  • For Account / Region click Search for regions and select your vSphere Datacenter
  • Enter a Storage Profile Name : vSphere Default Storage Profile
  • Select Storage Policy : Datastore Default
  • Enter a Capability Tag : Env:Production  and Silver This is again because we are using vSphere as our production environment and leveraging the default Disk.

cas-47

Azure Storage Profile

  • In Cloud Assembly, navigate to Infrastructure > Configure > Storage Profiles
  • Click + NEW NETWORK PROFILE
  • For Account / Region click Search for regions and select your Azure Region
  • Enter a Storage Profile Name : Azure Storage Profile
  • Select Storage Type : Managed Disks
  • Select Disk Type : Standard LRS
  • Select OS and Data disk caching : None
  • Select Supports encryption.
  • Enter a Capability Tag : Env:Testing  and Gold This is again because we are using Azure as our testing environment and leveraging the HDD backed Disks.

cas-48

Once completed you should have 3 Storage Profiles for the 3 cloud platforms ( vSphere, AWS, Azure ).

cas-49

Create A Project

Projects control who has access to Cloud Assembly blueprints and where the blueprints are deployed within the project. You use projects to organize and govern what your users can do and where they can deploy blueprints in your cloud infrastructure.

Cloud administrators setup projects, adding the required users and cloud zones. Anyone who creates and deploys blueprints must be a member of a least one project.

cas-50

As you can see, projects are simply groups that link users to cloud resources, controlling who can use what resource. Users become project members.

To deploy a blueprint , the deploying user must be a member of a project and the project must have one ore more cloud zones that support the development goals of the members. When the blueprint is deployed, the resource requirements defined in the blueprint as evaluated against the available zones and the blueprint is deployed to the cloud zone that supports those requirements.

How Can You Use Projects

You use projects in the way that best suits your users development goals.

  • Create a single project for a development team
    • The project includes a project administrator, the development team members, and all cloud zones that support the team workflow from development to testing to staging to production. the cloud zone capability tags we setup earlier target the zones where the blueprints are deployed.
  • Create multiple projects for a development team.
    • The project members might consist of all the same users or the membership might vary by role. For example, developer members for the development project, developers and testers for the testing and staging project, and lead developers for the production project.

Enough Talking lets go ahead and create a PROJECT

  • In Cloud Assembly, navigate to Infrastructure > Configure > Projects
  • Click + NEW PROJECT
  • Enter the following details for your prject
    • Name : VMWLAB Project

cas-51

    • Under Users, Click + ADD USERS. Here I m adding my self as an Administrator and clicking ADD

cas-52

    • Under the Provisioning tab > Cloud Zones add all the cloud zones we have created and give them a priority number.

cas-53

    • Click CREATE

Notice the Priority numbers we defined for each Cloud Zone when we added them, this mean if no capability tags were defined in a blueprint everything should go to AWS first, Azure second and vSphere Third.

Once completed you should see all the projects you created.

cas-54

Blueprinting

The Market place within Cloud Automation Services is a great way to quickly get started with blueprinting in Cloud Assembly. Not only do we provide several popular applications for deployment via Cloud Assembly, these blueprints also serve as example content you can learn from for how to complete complex tasks in Cloud Assembly YAML interface.

In order to get started with the market place we nee to first bind a My VMware Account

  • In Cloud Assembly, navigate to Infrastructure > Connections > Integrations
  • Click + ADD INTEGRATION
  • Select My VMware

cas-55

 

  • Enter your username and password and Click VALIDATE
  • Provide a name

cas-56

  • Click ADD
  • Select the Market Place Tab, you will be able to see the available sample blueprints. We can choose to either
    • Import the Blueprint directly into a Project
    • Download the Blueprint YAML file
  • For our blog I will be importing the Multi-Tier Web Application ( Word Press ) on On-Demand VMWare NSX-T Virtual Network listed under Technologies > Application Development.

cas-57

  • Clicking Open will give us a chance to read the summary of what this blueprint is about and more importantly the Tech Specs tab will tell us all the requirements for how we should be configuring the Blueprint and what Images to actually use for example
  • Click GET

cas-58

  • You may need to Read and Agree to the terms of a license agreement after that click NEXT to continue.
  • Here we will select to import it directly to our Project cas-59
  • Now we can switch to the Blueprint Tab to validate that the blueprint has been added to the project.

cas-60

  • Let’s Click on the Blueprint link to view its contents and observe the blueprint we imported from the marketplace.

cas-61

Here you go how awesome is that, infrastructure as code at our finger tips in a matter of seconds.

Blueprints from the market place will have temporary sample values assigned to them for the image, flavor, disk and network mappings. These will need to be updated with your own values of-course before attempting to do a deployment. For the propose of this blog we are simply demonstrating the existence of these blueprints to tweak and to learn from.

Creating and Deploying a Single Machine Blueprint

Here will be deploying a single-machine cloud agnostic blueprint based on all of our pervious configuration.

  • In Cloud Assembly, navigate to Blueprints 
  • Click + NEW 
  • Enter the following information, then Click CREATE:
    • Name: Ubuntu Small
    • Description: Ubuntu Small Cloud Agnostic Blueprint
    • Project: VMWLAB Project

cas-62

  • Drag a Cloud Agnostic machine to the canvas. Cloud Agnostic objects are designed to be portable between all supported cloud environments. This includes vSphere, AWS, GSP and Azure. Object types includes Machines, Networks, Load Balancers, and Disk Volumes.
  • In the code and under resources set the following :
    • Image as Ubuntu
    • Flavor as Small

cas-63

  • Click DEPLOY

cas-64

  • Select Create a New Deployment, then fill the required fields:
    • Deployment Name
    • Blueprint Version
  • Click DEPLOY and Monitor the status of the your request by navigating to the Deployment tab.

cas-65

  • When Provisioning is completed, view the deployment details by clicking on the deployment name and lets note where the machine was provisioned

cas-66

As you can see the machine was provisioned in AWS because if you remember, AWS Cloud Zone had the first priority when we added it during the creation of our project VMWLAB Project.

To figure out why AWS was selected we can also check the History Tab within the Deployment details where we can check all the Events for Requests for this deployment

cas-67

Here we can click on the Provisioning diagram shortcut for the ALLOCATE_FINISHED Event. This is will take us to Infrastructure > Activity > Requests to view the Request Details and see the various placement decisions made based on your Blueprint details.

Policy Based Placement

Multi-Cloud blueprints are capable of being moved between multiple environments leveraging tags to dictate their desired location via the Policy Based Placement Engine. In this section we will create a blueprint that is able to move between multiple cloud environment.

We have already created tags on the Cloud Zone Level that addresses the three environment we have Env:Development, Env:Testing, and Env:Production

We will go ahead and create additional tags against our environment but this time it will be based on our Cloud Platforms and on the compute Level for each of the platforms.

Note : Initially I have placed Capability Tags ( cloud:vspherecloud:aws, cloud:azure ) on the Cloud Account Type Level thinking I might need it but now I see more value in removing them from the Cloud Accounts level and instead setting them up at the Compute level within each of the Cloud Zone Type we have created. 

Configuring Tag Policies For Placement

For each of the cloud zones we have we will create a Capability Tag on the compute level. I will document here how we do it on the AWS Cloud Zone and then apply the same steps on the remaining Cloud Zones.

  • In Cloud Assembly, navigate to Infrastructure > Configure > Cloud Zones
  • Select the AWS Cloud Zone by Clicking OPEN 

cas-68

  • Select the Compute Tab and check the box for the Regions you want to use, here I am selecting both Availability Zones.

cas-69

  • Select TAGS  and enter the name of the tag cloud:aws under Add tags field then click SAVE

cas-70

  • Verify that the tag has been applied to the both compute resources then click SAVE to compete the task.

cas-71

  • Will complete the same process on the remaining cloud zones Azure and vSphere, instead we will be leveraging cloud:azure and cloud:vsphere respectively.

Placing the tag at the compute level is a common user case for customers to separate clusters within an environment based on a use case. An abstract version of this concept exists in public cloud as well ( People may use different regions/zones for different user cases).

We might tag a cluster designed for Oracle workloads to leverage the app:oracle tag, allowing us to place these workloads on this cluster via the placement engine. Another use case is for compliance reasons where users may tag clusters based on compliance capabilities on specific environments to ensure workloads land in an environment that will help them pass audits.

Creating Multi-Cloud Blueprint

Let’s create a multi-cloud blueprint leveraging our basic tag placement set, we can now create a blueprint that leverages these tags as part of the placement.

  • In Cloud Assembly, navigate to Blueprints 
  • Click + NEW 
  • Enter the following information, then Click CREATE:
    • Name: Multi-Cloud
    • Description: Multi-Cloud AWS/Azure/vSphere Blueprint
    • Project: VMWLAB Project

cas-72

  • Similar to our single machine blueprint, will drag a Cloud Agnostic Machine Object onto the canvas and configure it with an image type of Ubuntu and flavor of Small

cas-73

  • Under inputs we will add a new field as a drop down for our cloud environment. We can accomplish this with the following YAML:
inputs:
  cloud:
    type: string
    enum:
      - 'cloud:aws'
      - 'cloud:azure'
      - 'cloud:vsphere'
resources:
  Cloud_Machine_1:
    type: Cloud.Machine
    properties:
      image: Ubuntu
      flavor: Small
      constraints:
       -tag: '${input.cloud}'

This instantiates our menu to have AWS, Azure and vSphere as drop down menu options.

  • Also notice that we updated the Resources sections to include the constraints filed, referencing the tag properties, and a variable ‘${input.cloud}’ that references our drop down menu item. So what we select from the drop down menu will be the constraint tag that will decide the placement of the requested machine.

cas-74

  • Select DEPLOY and name the deployment and click Next to observe the Tagging option. Select the your choice and press DEPLOY

cas-75

  • After a few moments our deployment to AWS completes to our cloud environment.

cas-76

  • Select CLOSE  and return to Blueprints. Initiate another deployment and select any other Cloud environment like Azure for example .

cas-77

  • After a few moments our deployment to Azure completes to our cloud environment.

To summarize, we have created a Multi-Cloud Blueprint leveraging the YAML infrastructure as code and we presented to the requester a drop down menu based on Tagging constraints to select which Cloud environment he wants to deploy the machine to.

cas-78

Thank you very much if you have made it this far. I m hoping part 1 of this blog was beneficial and worth your time in exploring what you can do with Cloud Assembly .

In part 2, will explore more advanced topics such as Customizing Blueprints with Cloud-Config, versioning and Iterating on Blueprints.

The End, Eh!

 

 

 

 

 

Automation and Orchestration Blueprinting CAS Cloud Automation Services Machine Blueprints