How to Deploy vRA 8.0.1 while dealing with the Built-in containers root password expiration, preventing installations for vRealize Automation 8.0 and 8.0.1

Let’s get into it right away.

A few weeks ago the 90 days account expiry from vRealize Automation 8.0 and 8.0.1 GA releases has been exceeded for both the Postgres and Orchestrator services which runs today as Kubernetes pods.

This issue is resolved in vRealize Automation 8.1 which is soon to be released as of the writing of this post. ( Generally available in 1H20 ).

This issue is also resolved in Cumulative Update for vRealize Automation 8.0.1 HF1/HF2 so if you already installed the HF1 patch a while ago and before the account expiry, you have nothing to worry about.

But what about existing deployments that was not updated with HF1 or HF2 as of yet or net new deployments of vRealize Automation 8.0/8.0.1 and how they may be impacted by this issue. In this blog I address those scenarios in terms of what needs to be done to continue benefiting from everything the automation solution have to offer today and/or have a successful deployment when you do choose to deploy the vRealize Automation 8.0.1 solution until vRealize Automation 8.1 is released then you really don’t have to worry about any of this.

So let’s get started eh!.

Existing Deployments

For existing vRA 8.0 or 8.0.1 customers with active working instances, you have two options before you can reboot the appliance or restart the vRA services:

Option 1

Apply the workaround mentioned in KB 78235 and stay at vRA 8.0.1.

Scenario 1 : vRealize Automation 8.0/8.0.1 is up and running

  1. SSH into each of the nodes
  2. Execute vracli cluster exec -- bash -c 'echo -e "FROM vco_private:latest\nRUN sed -i s/root:.*/root:x:18135:0:99999:7:::/g /etc/shadow\nRUN sed -i s/vco:.*/vco:x:18135:0:99999:7:::/g /etc/shadow" | docker build - -t vco_private:latest'
  3. Execute vracli cluster exec -- bash -c 'echo -e "FROM db-image_private:latest\nRUN sed -i s/root:.*/root:x:18135:0:99999:7:::/g /etc/shadow\nRUN sed -i s/postgres:.*/postgres:x:18135:0:99999:7:::/g /etc/shadow" | docker build - -t db-image_private:latest'
  4. Execute opt/scripts/backup_docker_images.sh to persist the new changes through reboots.

Scenario 2 : vRealize Automation 8.0/8.0.1 is already down as a result.

  1. SSH into each of the nodes
  2. Run opt/scripts/deploy.sh --onlyClean on a single vRA node to shutdown the services safely.
  3. Once completed, Repeat step 2 through 4 in Option 1 – > Scenario 1
  4. Run /opt/scripts/deploy.sh to start the services up.

Option 2

Apply the vRealize Automation 8.0.1 HF1 or HF2  with vRealize Lifecycle Manager 8.0.1 patch 1

Scenario 1 : vRealize Automation 8.0/8.0.1 is up and running

It is recommended to install vRealize Suite Lifecycle Manager 8.0.1 patch 1 before vRealize Automation 8.0.1 patch 1. The vRealize Suite Lifecycle Manager 8.0.1 Patch 1 contains a fix for some intermittent delays in submitting the patch request.

Apply vRealize Automation 8.0.1 patch 1 leveraging vRealize Lifecycle manager 8.0.1 Patch 1.

Scenario 2 : vRealize Automation 8.0/8.0.1 is already down as a result.

  1. SSH into each of the nodes
  2. Run /opt/scripts/deploy.sh --onlyClean on a single vRA node to shutdown the services safely.
  3. Once completed, Repeat step 2 through 4 in Option 1 – > Scenario 1
  4. Run /opt/scripts/deploy.sh to start the services back.Apply vRealize
  5. Apply Automation 8.0.1 patch 1 leveraging vRealize Lifecycle manager 8.0.1 Patch 1

Note: We highly recommend to be always on the more recent builds and patches.

New Deployments

If you need a video tutorial on how to install vRealize Automation 8.x check either my Youtube video on how to deploy vRA 8.x with vRealize Easy Installer here or my previous blog post here which also include the video.

Please subscribe and smash that tiny notification bill to get notified of any new and upcoming videos if you do check my Youtube channel.

Now that is out of the way , for new deployments of 8.0.1 and until 8.1 is released where the issue is resolved, it is really very simple.

Once you see that vRA 8.0.1 is deployed via vRealize suite lifecycle manager 8.0.1 and that its now reachable via the network, do the following:

  1. SSH into the vRA node
  2. Execute Kubectl get pods -n prelude to see if vRA started to deploy a few of the services in the prelude namespace.
  3. Once confirmed proceed to step 4
  4. Execute vracli cluster exec -- bash -c 'echo -e "FROM vco_private:latest\nRUN sed -i s/root:.*/root:x:18135:0:99999:7:::/g /etc/shadow\nRUN sed -i s/vco:.*/vco:x:18135:0:99999:7:::/g /etc/shadow" | docker build - -t vco_private:latest'
  5. Execute vracli cluster exec -- bash -c 'echo -e "FROM db-image_private:latest\nRUN sed -i s/root:.*/root:x:18135:0:99999:7:::/g /etc/shadow\nRUN sed -i s/postgres:.*/postgres:x:18135:0:99999:7:::/g /etc/shadow" | docker build - -t db-image_private:latest'
  6. Execute opt/scripts/backup_docker_images.sh to persist the new changes through reboots.
  7. Keep checking the status of the pods by continually running and executing Kubectl get pods -n preludeuntil all the pods are up and running.

If your only installing one appliance and you noticed that the vco-app pod status is CrashLoopBackOff 

2020-03-28_14-00-53

You will need to delete the pod so a new one gets provisioned from the newly updated docker build that we generated in step 4 by executing the following below command.

kubectl delete pods -n prelude vco-app-pod-name

If your installing a cluster and since we can’t simply delete the postgres pod to fix it –So the other postgres instances on the remaining nodes are able to replicate data-otherwise other services that depends on postgres will also fail so its better to just shutdown all the services on each of the nodes and doing the following:

  1. SSH into the vRA node
  2. Execute Kubectl get pods -n prelude to see if vRA started to deploy a few of the services in the prelude namespace.
  3. Execute /opt/scripts/deploy.sh --onlyClean on each of the nodes to stop the services.
  4. Once completed execute the workaround repeating step 4 through 6
  5. Run /opt/scripts/deploy.shon each of the nodes to start the services up.

Once your appliance or cluster is up and running apply the vRealize Automation 8.0.1 HF1 or HF2. ( Soon to be also released ) as I mentioned above in Option 2 for Existing Deployment.

If you have already one appliance with HF1 you can’t scale out to create a cluster since the original image is not patched with HF1. So unfortunately you have to wait a couple more weeks until 8.1 is out, where then you can upgrade then scale out your deployment to create a cluster production ready deployment.

If you do have any questions please post them below. I will try my best to have them answered.

Hope this has been hopeful if you have made it to the end.

The End Eh!

Automation and Orchestration vRealize Automation

vSphere Customization with Cloud-init While Using vRealize Automation 8 or Cloud.

After spending an enormous amount of time, which I think started somewhere in the summer of last year to get vSphere Customization to work with Cloud-init while using vRealize Automation 8 or vRealize Automation Cloud as the automation platform to provision virtual machine deployments and install, configure the applications running on it.

I finally have a workaround that I can say is guaranteed to work every single time, until something better comes along. i.e. Software components like we have it today within the vRA 7.x platform, which is planned to be released for vRA 8.x in Q3-2020 if everything goes as planned.

With some out-of-the-box thinking, I was able to use IP static assignment ( assignment: static ) within the vRA blueprints to leverage the IP Static pool and the network metadata that we define in vRA via Network Profiles for the targeted networks we want to connect to, while using cloud-init with Ubuntu 16.04 and Ubuntu 18.04 for now, but the principle should be the same for other Linux distributions, even though it seems that RHEL is the only OS today that just works provided traditional Guest OS Customization GOSC is set in cloud-init.

Note: The will also work if you were to use DHCP IP Assignment.

Hoping this was worth the time, I am documenting in this blog the step by step instructions on how to prepare your vSphere templates while leveraging cloud-init,  in addition to for your own reference, a list of all the internet available resources that I looked at while doing my research.

I will also have a video added to the blog later that showcases going through the entire template preparation and also demo after that a typical vRA 8 deployment using static IP assignment while leveraging cloud-init to install selected packages per machine component and execute various commands to setup an application.

I still say that this shouldn’t be that hard for our customers to setup and hopefully Software Component like I mentioned would save us all from all this complexity, of-course this is beside the fact that you still can do this via various configuration management tools such as Ansible and puppet which by the way vRealize Automation 8 and cloud integrate with today out-of-the-box.

How it works?

In a high level when the virtual machine first boots up and gets rebooted to be customized due to the dynamic vCenter customization specs that gets created based on the fact we are using the assignment static property ( assignment: static ) within the blueprint code as you see in the screenshot below, I am making sure that during that time, Cloud-init is in a disabled state.

2020-02-15_11-26-33

After the customization reboot the virtual machine once, there is a Cron Job that I created on the template that execute at startup after a 90 sec of sleep which is enough time for the virtual machine to be customized, rebooted and connected to the network without running the Cron Job as of yet. After the initial reboot and pass the 90sec mark now the Cron Job execute a shell script that enables cloud-init and initializes it running all the needed cloud-init modules. ( init, Config and Final)

Note: Feel free to increase the 90 sec if you feel you need more time as the virtual machine being customized. 

The End result, the virtual machine is now customized with an updated host-name and an IP from our targeted static IP pool configured for the network its connected to without having to hack the Cloud Config code any further to setup things like the host-name or even configure the network itself, and more importantly without conflicting with cloud-init which what the problem was all along.

Let’s get started, Eh!

Template Preparation Steps

  • Once the virtual machine is up and running update the list of available packages and install any new available version of these packages that you have to update your template.

sudo apt-get update && sudo apt-get -y upgrade

    • Install Cloud-init for Ubuntu 16.04. Ubuntu 18.04 have cloud-init pre-installed so you can skip this step

sudo apt-get -y install cloud-init

  • Configure OVF as your Datasource, then save and exit

sudo dpkg-reconfigure cloud-init

  • Enable traditional Guest OS Customization GOSC Script by editing /etc/cloud/cloud.cfg file and adding

disable_vmware_customization: true

  • Ensure network configuration is not disabled in /etc/cloud/cloud.cfg, by deleting or hashing the following if it exists:

network:
config: disabled

Or that any /etc/cloud/cloud.cfg.d/* configuration files, such as
99-disable-networking-config.cfg with the folloing content “network: {config: disabled}” doesn’t exist.

  • Set Temp not to clear, by editing /usr/lib/tmpfiles.d/tmp.conf  and adding the prefix # to line 11.

#D /tmp 1777 root root –

  • Configure Open-vm-tools to start after dbus.service by editing /lib/systemd/system/open-vmtools.service file and adding the following under the [Unit] section.

After=dbus.service

  • Reduce the raise network interface time to 1 min by editing /etc/systemd/system/network-online.targets.wants/networking.service file and changing: ( This not applicable on Ubuntu 18.04 )

TimeoutStartSec=5min to TimeoutStartSec=1min

  • Disable cloud-init on First Boot and until customization is complete by creating this file /etc/cloud/cloud-init.disabled

sudo touch /etc/cloud/cloud-init.disabled

  • Create a script your_script.sh in a known location that will be called by a Cron Job that will create later to enable and initialize cloud-init after the customization reboot. The script should contain the following commands:
sudo rm -rf /etc/cloud/cloud-init.disabled
sudo cloud-init init
sleep 20
sudo cloud-init modules --mode config
sleep 20
sudo cloud-init modules --mode final
  • Configure the script to be an executable

sudo chmod +x your_script.sh

  • Create a Cron Job that will run after 90 sec of sleep at boot by typing crontab -e and entering the following:

@reboot ( sleep 90 ; sh /Script_path/your_script.sh )

  • Copy the content below for the Template Cleaning script and create your_clean_script.sh. You can replace cloudadmin with your own user that you setup when you installed the Ubuntu OS
#!/bin/bash

# Add usernames to add to /etc/sudoers for passwordless sudo
users=("ubuntu" "cloudadmin")

for user in "${users[@]}"
do
cat /etc/sudoers | grep ^$user
RC=$?
if [ $RC != 0 ]; then
bash -c "echo \"$user ALL=(ALL) NOPASSWD:ALL\" >> /etc/sudoers"
fi
done

#grab Ubuntu Codename
codename="$(lsb_release -c | awk {'print $2}')"


#Stop services for cleanup
service rsyslog stop

#clear audit logs
if [ -f /var/log/audit/audit.log ]; then
cat /dev/null > /var/log/audit/audit.log
fi
if [ -f /var/log/wtmp ]; then
cat /dev/null > /var/log/wtmp
fi
if [ -f /var/log/lastlog ]; then
cat /dev/null > /var/log/lastlog
fi

#cleanup persistent udev rules
if [ -f /etc/udev/rules.d/70-persistent-net.rules ]; then
rm /etc/udev/rules.d/70-persistent-net.rules
fi

#cleanup /tmp directories
rm -rf /tmp/*
rm -rf /var/tmp/*

#cleanup current ssh keys
#rm -f /etc/ssh/ssh_host_*

#cat /dev/null > /etc/hostname

#cleanup apt
apt-get clean

#Clean Machine ID

truncate -s 0 /etc/machine-id
rm /var/lib/dbus/machine-id
ln -s /etc/machine-id /var/lib/dbus/machine-id

#Clean Cloud-init
cloud-init clean --logs --seed

#cleanup shell history
history -w
history -c

  • Configure the Template Cleaning script to be an executable as well

sudo chmod +x your_clean_script.sh

  • Make sure you can switch to user root by editing the fine /etc/ssh/sshd_config and changing PermitRootLogin to yes

PermitRootLogin yes

  • Set a password for root

sudo passwd root

Note: The reason for the above is to be able to execute the clean script with no issues as I personally had issues executing the clean up script with sudo working with the Ubuntu 18.04. you can always revert it back once the cleanup template is executed. 

  • Execute the Template Cleaning Script.

sudo ./Script_path/your_clean_script.sh

  • Shutdown the virtual machine and turn it into a template.

Shutdown -h now

Note : Just be aware that the cron job will also run if you try to update the template for any reason . So make sure if you do pass 90 sec while doing your change is to re-add the /etc/cloud/cloud-init.disabled file and then re-execute the clean up script again before shutting down the template .

if you dont cloud-init will execute on first boot and you will get the vm customization but your cloud config code wont be applied

Click To See It All In Action On my Youtube Channel !

 

Happy Template Building! Please share! 

The End Eh!

Resources:

https://ubuntu.com/engage/cloud-init-whitepaper
https://debconf17.debconf.org/talks/164/
https://cloudinit.readthedocs.io/en/latest/
https://events.linuxfoundation.org/wp-content/uploads/2017/12/Cloud-init-The-cross-cloud-magic-sauce_Smith_moser.pdf
https://www.youtube.com/watch?v=RHVhIWifVqU
https://www.youtube.com/watch?v=y8WA1BUlT-Q
https://linuxtechlab.com/executing-commands-scripts-at-reboot/
https://blogs.vmware.com/management/2019/02/building-a-cas-ready-ubuntu-template-for-vsphere.html
http://kb.vmware.com/s/article/56409
https://kb.vmware.com/s/article/59687
http://kb.vmware.com/s/article/59557
http://kb.vmware.com/s/article/2378666
https://blah.cloud/infrastructure/using-cloud-init-for-vm-templating-on-vsphere/
http://ubuntu.com/blog/cloud-init-v-18-2-cli-subcommands
http://lucd.info/2019/12/06/cloud-init-part-1-the-basics/

 

Blueprinting Cloud Automation Services Cloud-init vRA Blueprints vRealize Automation vSphere Customization

Part 3: vRealize Automation 8.0 Deployment with vRealize Suite Lifecycle Manager 8.0

In Part 2 of my vRealize Automation 8.0 blog video series, we have upgraded vRealize Lifecycle Manager 2.1 to 8.0 by performing a side by side migration leveraging the vRealize Easy Installer while importing the management of both VMware Identity manager 3.3.0 and the vRealize Suite 2018 environment.

In this blog video we will be using vRealize Lifecycle Manager 8.0 to deploy vRealize Automation 8.0 in a new environment.

Now as for requirements you will need :

  1. vRealize Lifecycle Manager 8.0
  2. VMware Identity Manager 3.3.1
  3. A new Hostname, IP Address and a DNS record for the new vRA 8.0 appliance that the vRealize Suite Lifecycle Manager 8.0 will be creating.
  4. Product Mapping is set with the install and upgrade binaries for the new vRealize Suite 2019 Products.

 

Deployment Workflow

2019-10-23_10-13-17

Please note that the installation process in the video after hitting submit is fast forwarded.

The End, Eh!

Automation and Orchestration vRealize Automation vRealize Suite vRealize Suite Lifecycle Manager

Part 2: Migration of vRSLCM 2.x Version to vRealize Suite Lifecycle Manager 8.0

If you happen to have an existing vRSLCM 2.x and vIDM 3.3.0 in your environment then you will need the vRealize Easy Installer to migrate your existing vRSLCM 2.x instance to vRSLCM 8.0.

Once your migration to vRSLCM 8.0 is completed you can upgrade your vIDM instance to 3.3.1 since its a requirement before you can install vRealize Automation 8.0 with vRealize Lifecycle Manager 8.0

Again as a reminder vRealize Automation 8.0 is installed, configured, managed and upgraded only through vRealize Suite Lifecycle Manager 8.0

Now as for requirements you will need :

  1. A new Hostname, IP Address and a DNS record for the new vRSLCM 8.0 appliance that the vRealize Easy Installer will be creating.
  2. To make sure that the password for the sshuser on the existing vIDM appliance is not expired.
  3. To enable root access for SSH on the existing vIDM appliance following VMware KB 2047626
  4. To Download the install and upgrade binaries for vRealize Suite 2019
  5. To Make sure you have enough storage on the new vRSLCM 8.0 appliance.

Migration Workflow

migration flow

Please note that the installation process in the video after hitting submit is fast forwarded.

NOTE (vIDM Upgrade Support )  :

  • Green-field vRealize Suite Lifecycle Manager 8.0 supports only 3.3.1 version of VMware Identity Manager to be installed or imported.
  • Older versions (2.9.2, 3.2.0, 3.2.0.1 & 3.3.0) of VMware Identity Manager will be supported only for existing vRealize Suite Lifecycle Manager instances that are being migrated to vRealize Suite Lifecycle Manager 8.0.
  • Upgrade support from older VMware Identity Manger to latest is only available if they conform to vRealize Suite Lifecycle Manager supported form-factor.
  • Versions prior to vRealize Suite Lifecycle Manager 8.0 allowed only single instance of VMware Identity Manager to be deployed with embedded connector and embedded postgresql database.
  • Upgrade for VMware Identity Manager within vRealize Suite Lifecycle Manager 8.0 to latest versions will only be supported if it conforms to the above mentioned form-factor.

Else the upgrade has to be performed outside vRealize Suite Lifecycle Manager and Once upgraded, it can any-time be re-imported by triggering Inventory Sync in vRealize Suite Lifecycle Manager 8.0

 

The End, Eh!

Helpful Links You Might Need

Resetting the admin@localhost password in vRealize Suite Lifecycle Manager

Restting root password on photon OS

vRealize Automation vRealize Suite Lifecycle Manager

Part 1: vRealize Automation 8.0 Simple Deployment with vRealize Easy Installer

On October 17th, 2019 VMware announced the next major release of vRealize Automation. it uses a modern Kubernetes based micro-services architecture and brings vRA cloud capabilities to the on-premises form factor.

What’s New

The many benefits of vRA 8.0 include:

  • Modern Platform using Kubernetes based micro-services architecture that provides
  • Easy to setup and consume multi-cloud infrastructure surface
  • Embedded vRO 8.0 Web Client and Orchestrator’s new release features
  • Deliver Infrastructure-as-Code using a declarative YAML syntax
  • Cloud Agnostic Blueprints
  • Iterative development of Blueprints
  • Self-service catalog coupled with agile governance
  • Collaboration across teams via sharing of objects
  • Kubernetes/container management
  • Deploy IPv6 workloads on dual-stack IP (IPv4/ IPv6) networks in vSphere
  • CI/CD pipeline and automated application release management
  • New Action based extensibility (ABX), which allows you to write lightweight scripts, using node.js and python.
  • Git Integration to manage all blueprints, workflows, actions and pipelines.

For more information, kindly refer to the Release Notes

vRealize Automation 8.0 is installed, configured, managed and upgraded only through vRealize Suite Lifecycle Manager 8.0 .

In the video posted below, I’am going to provide the step-by-step process of using the vRealize Easy Installer to :

  • Install vRealize Suite Lifecycle Manager 8.0
  • Deploy VMware Identity Manager 3.3.1 and register with vRealize Automation.
  • Install new instance of vRealize Automation 8.0

 

Installation Workflow

installer workflow

Please note that the installation process in the video after hitting submit is fast forwarded.

The End, Eh!

Automation and Orchestration vRealize Automation

vRealize Automation Extensibility Starts with SovLabs Plug-in – Part 1

When you start looking at vRealize Automation extensibility and how you can integrate it into your own datacenter ecosystem or how you can accommodate certain extensibility use cases like provisioning workloads with custom host names based on a business logic or as simple as running scripts or attaching tags post provisioning, you usually have 3 options :

  1. You can do it yourself. (High Time To Value, Local Skill Set)
  2. You can use Professional Services. ( High Time To Value , Expensive )
  3. You can use SovLabs Extensibility Frame work. (Instant Value, Production ready, Fully Supported, Off the shelf extensibility content and a Fraction of vRA cost)

SovLabs provides one common framework for extending VMware vRealize Automation (vRA) where you can replace custom orchestration workflow development with ready-to-run extensibility software. It brings extensibility into the SDDC framework where its :

  • Managed as a native component of the SDDC directly from vRA
  • Interoperable with latest releases and versions of vRA and endpoints
  • Prescriptive, each module comes ready to run.
  • Flexible, easy to modify without touching a single line of code
  • Enterprise support ( Production ready and Scalable )

In this blog we will explore, in two parts :

  • Part 1: How to install the Sovlabs Plug-in
  • Part 2: How to configure basic Sovlabs Modules like:
    • Custom Naming
    • Microsoft Active Directory
    • Microsoft DNS
    • Property toolkit
    • VM Tagging
    • ServiceNow CMDB

 

Part 1 : Installing Sovlabs Plug-in 2018.3.0

Here we will be doing a new install but before we do that we need to address a few prerequisites around vRA and vRO, so please read carefully :

vRA Prerequisites 

  • A Working instance of vRealize Automation 7.5 where you have successfully provisioned a VM from vRA using a blueprint.
  • Keep things simple according to the Sovlabs documentation by not using spaces or camel-casing for Tenant name, Users, Group Names or Business Groups. Not sure if thats the case for my vRA instance in my lab since its already setup but we shall see!
  • For Clustered vRA and/or vRO, load balancing is configured to VMware’s documentation. this is not our case here but for more details check the SovLabs documentation here HA and vRA with SovLabs.

Assuming you know your way around vRealize Automation you need to have the following:

Service Account User

  • You can create or use a local vRA user for the ownership of the SovLabs Endpoints, Profiles, and Services as well as connectivity into vRO to run workflows. your can create for example a new user called sov_admin but in our case we will be using the configurationadmin local user as a Service Account User, that got created during the content creation at the end of the vRA deployment.
  • Make sure the Service Account User has the following roles in vRA :  IaaS Administrator, Tenant Administrator, XaaS Architect.

pic1

pic2

vRA Custom Group

  • Create a Custom Group in vRA for the ownership of the SovLabs Endpoints, Profiles, and Services as well as connectivity into vRO to run workflows called sov_admins for example and make the configurationadmin Service Account User a member of this group.
  • Grant the sov_admins Custom Group both the Tenant Administrator and XaaS Architect Roles during the creating of the vRA custom group

pic4

  • Make sure that the Custom group sov_admins also have the IaaS Administrator role in addition to what we entitled it in the pervious step.

pic3

vRA Business Group

  • Login as the Service Account User configurationadmin@vsphere.local
  • Create or use a SovLabs specific vRA Business Group for allowing entitlements and ownership of SovLabs content to be confined to its own group. in our case we will be leveraging the Configuration Administrators Business Group, that got created during the content creation at the end of the vRA deployment.
  • Make sure to add the Sov_admins Custom Group to the Group Manger Role field within the Configuration Administrator Business Group.

pic5

vRO Prerequisites

  • vRO is already Setup and configured .
  • vRO may be embedded (recommended) like in our instance here or it can be also external. You can refere to VMware’s Install and Configure document.
  • vRO is Setup as an Endpoint in vRA, so click on the Infrastructure tab > Endpoints > Endpoints > Orchestration > vRealize Orchestrator to add your embedded or external vRO endpoint in vRA.

blog1

Modify Files and Set Permissions

  • Modify vmo.properties and js-io-rights files and set permissions. To do that SovLabs provides an script you can download from here called sov_vro_config.sh to automate the modification of those two files and it also creates the krb5.conf file for Kerberos Authentication.
  • Copy the sov_vro_config.sh script to your vRA/vRO appliance since we are using the embedded vRO, its actually the same appliance.
  • Make the script executable by executing the following command then execute the script and follow the instructions.
chmod +x sov_vro_config.sh
  • Restart the vRO service by executing the command
service vco-server restart

Create vRO vRA Host

  • Create vRO vRA Host via vRO Workflow as the default vRA host will not work fr the SovLabs plug-in. The vRA Host must be Shared Session mode and since we are using vsphere.local as our default and only tenant, the name to use for vRA host endpoint should begin with sovlabs_
  • Login to vRO using the vRO client and switch to the Design Mode

blog2

  • In the Workflows tab, go to: Library > vRealize Automation> Configuration > Add a vRA Host
  • Right-click the workflow and click Start workflow and fill out the form:

blog3

  • Click Next in the form wizard

blog4

  • Click Submit in the form wizard
  • New Inventory item for the vRA Host will be in the Inventory tab in the vRO client.

blog5

Create vRO vRA IaaS Host

  • Back in the Workflows tab in the vRO Client, go to: Library > vRealize Automation> Configuration > Add an Iaas host of a vRA Host
  • Right-click the workflow and click Start workflow and fill out the form:
  • Choose the vRA host in the list that we created in the previous step

blog6

blog7

  • Click Next for Host Properties and accept the defaults. The fields should all be auto-filled

blog8

  • Click Next for Proxy settings

blog9

  • Click Next for User credentials. Since we are using the embedded vRO, we will be selecting SSO and click Submit

blog10

  • New Inventory item for the vRA IaaS Host will be in the Inventory tab in the vRO client. You may need to refresh the vRO Client

blog11

Installing the SovLabs Plugin

  • Assuming your already download the SovLabs plugin along with the license key  provided to you in an email from SovLabs its time to install the plugin.
  • Login to the vRO Control Center as user root https://%5BvRO-FQDN%5D:8283/vco-controlcenter/ .  Make sure that the Orchestrator user interface is started and running.

blog1

  • Click on the Manage Plug-Ins icon

blog2

  • In the Install plug-in section, browse for the SovLabs Plugin file (o11nplugin-sovlabs.vmoapp)
  • Click Upload

blog3

  • Accept the EULA and click on Install

blog4

blog5

  • Restart the vRO Server or wait for the server to restart on its own
    • On the Home page, click on the Startup Options icon and click on Restart
    • Optionally, SSH to the vRO appliance and type in: service vco-server restart
  • Click on theManage Plug-Ins icon and Verify that the SovLabs Plugin is listed among the installed vRO plugins

blog6

  • Restart vRA

Configure the SovLabs Plugin

  • Login to vRO using the vRO client and switch to the Design Mode if you haven’t already.
  • In the Workflows tab, go to SovLabs > Configuration folder and expand it.
  • Right-click on the SovLabs Configuration workflow and click Start Workflow.
  • Accept the EULA by selecting Yes and click Next

blog7

  • Under the Service and Content :
    • Choose the appropriate tenant = vsphere.local
    • Choose the SovLabs Business Group= Configuration AdministratorsRemember Earlier I mentioned that will  be using this BG for SovLabs
    • Select Yes to Create SovLabs vRA Catalog serviece
    • Choose the SovLabs vRA Custom Group we created earlier, that will be your security group = sov_admins@vsphere.local
    • Select Yes to Publish License Content
    • Click Next

blog8

  • In the Upgrade Options section of the form: Select No and click NextSince this isn’t an upgrade
  • In the Install/Update SovLabs Workflow Subscriptions section of the form: Select Yes. This will create all the needed Event Broker Subscriptions in vRA
  • Click Submit
  • Once completed you should see green check on SovLabs Configuration Workflow and for both its Sub Workflowsblog9

Add SovLabs Latest License

For the SovLabs latest Plugin to works it needs a 2018.x.x license key. Here are the steps to add the License Key

  • Login to the desired vRA tenant which is in our case is vSphere.local and login using the Service Account User configurationadmin@vsphere.local

Totally Optionals  :

    • In my case and because I want to continue to use my main account cloudadmin@vmwlab.local user, I simply granted the User Role to the cloudadmin user in the Configuration Administrators Business Group
    • Then added the Cloudadmin user in the SovLabs vRA Extensibility Modules Entitlement created  by the SovLabs plugin.
  • Now that I have access I can Click on the Catalog tab, then Click on the catalog item Add License – SovLabs Modules

blog10

  • Click Request and Copy and paste the provided license key and click SUBMIT

blog11

Note : The Screenshot shows a fraction of the license key, not the entire license key.  🙂

  • You can Monitor your In Progress request in the Deployments Tab

blog12

  • After the license is successfully added, SovLabs Catalog Items and SovLabs vRA Event Broker Subscriptions will appear/generate.

 

Thank you very much if you have made it this far, in part two will touch on How to configure basic Sovlabs Modules like:

  • Custom Naming
  • Microsoft Active Directory
  • Microsoft DNS
  • Property toolkit
  • VM Tagging
  • ServiceNow CMDB

Please feel free to comment or provide feedback

The End of Part 1 Eh!

Automation and Orchestration Extensibility vRealize Automation

Installing and Configuring the vRealize Automation 7.5 (vRA 7.5) ITSM 5.0 / 5.1 Plug-in for ServiceNow

A  new VMware vRealize Automation plugin 5.0 was released on November 2nd on the VMware market Place Link for Servicenow that provides an out of the box integration between the Servicenow portal and vRealize Automation 7.5 catalog and governance model. It enables ServiceNow users to deploy virtual machines using vRA 7.5 and perform basic ServiceNow day 2 operations on their CMDB assets.

Update : There is now an updated version of the ITSM plug-in 5.1 that was released right after and currently available on the VMware Market Place Link

Key Features

  • Enables ServiceNow to integrate vRealize Automation 7.5 with ServiceNow platform and provide the ability for ServiceNow users to access the vRA catalogs, resources within ServiceNow.
  • The integration will allow end users to Request vRA catalog items from ServiceNow portal.
  • The plugin will fetch categories, catalog items and resource data from vRA platform and extend ServiceNow functionality to be able to render vRA catalog items into ServiceNow dynamically and manage vRA resources.
  • Day2 operation actions performed in ServiceNow CMDB will be updated back to vRA platform by giving API calls to vRA.
  • The plug-in supports vSphere, Amazon, Azure, and XaaS virtual machine provisioning, including formless and form based day 2 operations

In addition to all the above generic key Features, the ITSM 5.0 plug-ing includes fixes and new features such as :

  • Easier plug-in configuration through service account and Servicenow based RBAC and Entitlements
  • Multi-vRA support
  • Day 2 operations Enhancements
  • vRealize Business field support
  • ADFS or SSO setup are not required
  • Improve Login process for ServiceNow users with seamless authentication/entitlement model
  • Does not require end user access to internal vRA portal
  • Fully supported by VMware Global Services Support – GSS

In this blog we will take a look at how to deploy and configure the newly anticipated ITSM 5.0/ 5.1 Plug-in for vRealize Automation 7.5.

Update : Here are the fixes that were provided in the updated ITSM 5.1 plug-in:

  • Dynamic dependent drop-down fixes
  • Size, Image profile fixes
  • Disk fixes for null error
  • Token Encryption
  • Improved Entitlement Module
  • Access control fixes(ACL)
  • Duplicate catalog item form section fixes

In addition to this, there is scope change in V5.1 compared to V5.0 to avoid collision with V4.0.

So let’s get started, Eh!

Step 1: Prerequisites

The ITSM 5.0 plugin is targeted for vRealize Automation version 7.5. ITSM plugin interacts with vRealize automation using MID server. MID server is an IaaS component (deployed on prem – in the same network as vRA 7.5) having installed binaries provided by Service Now. For enabling the MID server component – Service now instance should be registered in MID server.

The ITSM 5.0 Plugin is compatible with the following ServiceNow releases (Jakarta, Kingston, London). After registering the vRA instance on Service now portal, data collection needs to be done to fetch all the required vRA contents (like Catalog).

Once the catalog is imported to Service now, a user can place requests from the catalog based on their entitlements.

You will need :

  • Download a copy of the ITSM Plugin 5.0 from VMware market place.
  • A ServiceNow Instance – Jakarta, Kingston orLondon release.
  • A MID Server established and connected to your ServiceNow Instance.

If your looking for how you can do that, please reference my pervious blog on ITSM 3.0 blog

Blog1

  • A vRealize Automation 7.5 instance configured on prem where you have configured and tested  one or more blueprint deployments successfully.

Make the plug-in accessible from all application scopes

To do that we need to navigate to Script Includes by using the search from the left navigation menu. Then Selecting System Definition > Script Includes

Once you select the Script Includes, do a search for JSUtil on the right page for the Name field. Once you find the script Open it by clicking on it.

Blog11

On the Accessible from drop-down, select All Application Scopes then select Update. Ensure that the changes are saved.

Blog12

Enable Application Access on Tables

You must enable application access to certain tables for the plug-in to work. Repeat the following steps on all the Tables below to modify:

  1. user_criteria
  2. sc_category_user_criteria_mtom
  3. item_option_new
  4. catalog_script_client
  5. question_choice
  6. catalog_ui_policy
  7. catalog_ui_policy_action
  8. sc_cat_item_user_criteria_mtom
  9. sc_req_item
  10. sc_category

Option 1 : Procedure To Enable Application Access

  • Log in to ServiceNow as an administrator.
  • Search for System Definition  in the filter navigator and click Tables
  • Search for each table in the Name filter on the right.

Blog13

  • Click on the Table Label under the Label column that matches the Table Name search you did. All the records are in the Global Application mode, you will need to click the option on the top of screen to edit the record. That is if your were still asked, usually you shouldn’t have to since we enabled Global access from all application scopes in the pervious step.
  • Click Application Access.
  • Select the can read, can create, can delete, and can update check boxes for each table.
  • Click Update and Repeat.

Blog14

Option 2 : Procedure To Enable Application Access

You may find this way faster to update all the tables listed, please watch the video to do so. You will be adding .list at the end of the table name and using the Filter navigator to search for it

Step 2: Installing the ITSM 5.0 Plug-in

  • Log in to the ServiceNow portal as a system administrator.
  • Select System Update Sets > Retrieved Update Sets > then select Import Update Set from XML

Blog2

Click Choose File on the dialog to choose the file to upload, and then select the VMware-vRealize-Automation-Application-ITSM-V5 file > Click Upload.

Blog3

In the Retrieved Update Sets list, select the VMware vRealize Automation Application ITSM V5.0 update set in the Name column by clicking on it once its in a Loaded State.

Blog4

 

Select Preview Update Set to validate the update set before committing it.

Blog5

A dialog box confirms update set validation.

Blog6

Click Close and review the errors

In my testing I was using the ServiceNow London Release so I encountered 4 errors as you can see in the screenshot below. From pervious experience I was told many times that these records existed in the instance where the plug-in was developed and thats why we are receiving these error during the validation since these records don’t really exist in our instance.

go ahead and click on Accept Remote Update on each of the errors.

Blog7

If you are using the ServiceNow Jakarta release and ServiceNow displays the below error message, click Accept remote update as well.

"Could not find a record in sc_homepage_renderer for column homepage_renderer referenced in this update"

Once you have accepted all remote updates click Commit Update Set

Blog8

A dialog box opens automatically after you click Commit Update Set while the commit action is in progress. When its done click the Close button when it appears to dismiss the dialog. It took 25 minutes to complete so please be patient.

Blog9

From the left menu, Click Update log

The install is complete when a message appears stating Finished update load from database but you can continue on at this point as long as you can see that the state of the Plug-in Update Set is Committed.

To do that select Retrieved Update Sets in the left menu and verify that the update set has a status of Committed.

Blog10

*  Important Note Only if your deploying the ITSM 5.0 plug-in 

After the installing is complete, search for Integration – vRealize Automation > then select Administration > System Properties 

On the System Properties page Search for the Name and Change the value of the x_vmw_vmware_vrasp.vrasn.group.assignment_group System Property to > d64ea542db920300435fd001cf961913

This is the sys_id of the group which is for approval of requests within ServiceNow. The value was wrongly captured in the 5.0 final build therefor it was documented to change its value.

*  Again this is not needed if  your installing the ITSM 5.1 version of the plug-in

Step 3: Users Facts and Setup

  • The Plugin configuration can be done by a system administrator like I m doing in this blog or by a user with x_vmw_vmware_vrasp.vrealize_automation_catalog_admin privileges.
  • You will need to Add the role x_vmw_vmware_vrasp.vra_user in ServiceNow for all users that must access vRealize Automation, to enable those users to see the vRealize Automation User Portal module which will we will cover later in the blog, including admin,catalog admin, and end user
  • RBAC in ITSM 5.0 is independent from vRealize Automation RBAC.
  • Login and Authentication rules do not require validation from vRealize Automation side. All roles and entitlements are based on the ServiceNow model.
  • Approvals can be generated if the users have the x_vmw_vmware_vrasp.vrealize_automation_catalog_admin role and are a member of the vRealizeAutomation-ApprovalManagersGroup group in ServiceNow.
  • The plug-in admin role x_vmw_vmware_vrasp.vrealize_automation_catalog_admin must contain the “catalog_admin”, “itil” and “agent_admin” roles out of the box in order to see and configure the Mid Server module from the left pane.
  • The Plugin end users role x_vmw_vmware_vrasp.vra_user must have the “itil” role out of the box.

For my testing purposes and based on the information I just mentioned I granted all the roles to the System Administrator. Of course if your doing this in production you would be selective in terms who have access to these roles.

In Filter Navigator search for System Security > Users and Groups > Users and edit the System Administrator role membership so it includes :

Blog15

Step 4: Update the vRealize Automation Workflow for Requested Items

The system admin can configure the vRA Workflow for Requested Item using the workflow editor. At a minimum, you must assign the approval group that contains your approvers. When users request vRealize Automation catalog items, this workflow runs, and approvals are sent to the approvers within the approval group before the request is submitted to vRealize Automation.

By default the vRealizaAutomation-ApprovalManagersGroup group is set as an approving group in the workflow. You can change the approval group by the procedure below.

The approval group must contain the x_vmw_vmware_vrasp.vrealize_automation_catalog_admin role.

Follow the steps below if you want to use your own approval group and add it to the vRealize Automation workflow.

  • Search for Workflow Editor in the ServiceNow navigation pane and click it.
  • Search for vRA Workflow for Requested Item and open it.
  • Click the menu button and select Checkout.
  • Double-click the Approval group stage in the workflow.
  • Click the Edit Groups button.
  • Search the list of groups and make the appropriate selections.
  • Lock your selection by clicking the Lock icon.
  • Click Update.
  • Click the menu button and select Publish.

Again for my testing I edited the existing default group vRealizaAutomation-ApprovalManagersGroup so it has the x_vmw_vmware_vrasp.vrealize_automation_catalog_admin role and added the System Administrator as a member by searching in the Filter Navigator for System Security > Users and Groups > Groups and editing the group roles and membership accordingly.

Blog17Blog16

Step 5: Set up the Integration User

You must set up a vRealize Automation integration user. ServiceNow requires this user to import catalog items, categories, request statuses, and resources from vRealize Automation. In order to import items, the integration users must be a business group manager within the business groups that you want ServiceNow to manage. The integration user does not require a role within ServiceNow.

  •  Log in to vRealize Automation as a business group manager.
  • Edit your business groups and assign the integration user as a business group manager.

In our case here I have a business group in vRA 7.5 called Cloud Administrators and I have their AD security group listed as a member of the Group Manager Role. 

Blog18

For our testing will be using the CloudAdmin user which is a member of the Cloud Administrators Group.

Step 6: Register a vRealize Automation Instance in Service Now

With ITSM 5.0 you can register multiple vRealize Automation instances and use catalog items from all of them in one ServiceNow portal.

Note : All requests from ServiceNow for a specific vRealize Automation instance are placed in the name of user registered under the Register vRA module. 

Procedure To Register a vRealize Automation Instance

  • Log in to Service Now as a plug-in admin.
  • Navigate to and click the Register vRAs tab under Integration – vRealize Automation.
  • Click New.

Blog19

  • Enter the details of your vRealize Automation instance and click Save

Blog20

  • That takes you back to the Register vRAs menu where you see vRA instance you just configured.
  • We need now to Import services and catalog items from the vRealize Automation instance to Service Now, so lets click on the instance.

Blog21

  • Click Import Services and Catalog Items and monitor the import

Blog22

  • You can refresh the page with completed items by clicking List controls in the top left corner of the page and selecting Refresh list until there are no records to display before you move on. .
  • Next will repeat the process by clicking on the Register vRAs > Our vRA instance but this time we will Import and reconcile CMDB from your vRealize Automation instance to Service Now.

Blog23

  • Again you can refresh the page with completed items by clicking List controls in the top left corner of the page and selecting Refresh list until there are no records to display before you move on.

Blog24

Remember that you can always add more vRA Instances or update/ delete your current ones.

Update : Once you’re done, there are some scheduled imports that needs to run before you start using the plug-i.  If you don’t run them manually, the scheduled imports will run at there interval time setting but then you will have to wait until they all run.

To speed things up you need to execute the schedule imports in the right order :

  • Log in to Service Now as a plug-in admin.
  • Navigate to and click the Register vRAs tab under Integration – vRealize Automation.
  • Click on the Scheduled Imports 
  • Run each of the imports in sequence starting with 1  by clicking into each of the scheduled imports and clicking on Execute No, until you run them all.
  • Verify that there are no records within the Scheduled Imports Queue before starting the next Import.

Step 7: Create ServiceNow Entitlement

Here we are going to assign access for services, items, and actions by entitling users and groups in ServiceNow regardless how they are entitled in vRealize Automation.

These Entitlements in the vRealize Automation plug-in for ServiceNow are based on ServiceNow plug-in implementation and are unrelated to vRealize Automation entitlements at all.

Procedure To Create New Entitlements

  • Navigate to and click the vRA Entitlements tab under Integration – vRealize Automation.
  • Click New.

Blog25

  • Enter a name and description for the entitlements.
  • Select the user or group to entitle.
  • Select the services, items, and actions you want to entitle.

Update : In my case I have created a user called Scott Smith and granted him the x_vmw_vmware_vrasp.vra_user and the Itil Role. These are the minimum roles for a service now user who needs to access the vRealize Automation Portal in Service Now.

I also selected the vsphere Services Category and one of the items in it ( CentOS7.5 – ServiceNow Testing ) Bluepint and some of the supported Actions

You can click on the search icon to see a full list of services, items, or actions and you can unlock the pad lock to edit your selection for each such section and use the pad lock to lock it down.

Important Note : Not all the actions are supported even though its available in the UI and based on the documentation, Here is what is really supported :

  • Deployment Actions : Destroy and Expire
  • Item Actions : Suspend, Power On, Power Off, Shutdown and Expire
  • Click Submit when your done . As you can see I didn’t select any Services but I selected one basic vRA Blueprint that I wanted to Entitle my user Scott Smith to.

Blog31

Step 8: Request a Catalog Item

You can request a catalog item from the vRealize Automation user portal. Depending on your vRealize Automation plug-in configuration you might have identical catalog items from different vRealize Automation instances. For environments with multiple vRealize Automation instances, select from which instance you want to request the catalog item.

Procedure to Request a Catalog Item

  • Login to ServiceNow Portal as Scott.Smith
  • Navigate to and click the vRealize Automation User Portal tab under vRealize Automation Module that will open a new tab where you can access the portal

Blog27

 

  • In the vRealize Automation user portal, click Catalog Items.
  • Select the vRealize Automation instance, from which you want to request a catalog item

Blog32

  • Select a category and click Request on the catalog item.

Blog33

 

  • Enter the details of your request if any and click Submit.

Blog34

  • You are redirected to the Activities tab where you will see its awaiting approval

Blog35

  • You can click on the Request to find more details like the Stage or the State of the request . Once approved by the Approval group where the System Administrator happen to be a member in our case here.
  • For the Approvers to approve any of the requests they also can also go to the vRealize Automation Portal in ServiceNow and click on the Activities Tab > Approvals, find and click on the request that is Awaiting Approval and Approve or Reject the request.

Blog36

  • Once Approved, our user Scott Smith can see that its approved in his own portal Under the Activities Tab > Requests

Blog37

  • On the vRealize Automation side of things we can see that the request Blueprint is being provisioned

Blog38

  • The ServiceNow user can continue to track the machine request status through the Activities Tab  until the request is complete and closed in ServiceNow.

Blog39

  • If your user is entitled, you can make changes to your deployments and virtual machines after they have been created.
  • Your user must have the specific entitlement that corresponds to the action you want to make. From the Actions tab you can power on, restart, expire, destroy, power off your deployment, and more.

Blog40

Hope you found the blog around the new ITSM 5.0 Plug-in beneficial if you have made it this far. This was a quick introduction around the plug-in installation and configuration, of course there will be more things that need testing as I continue to use the Plug-in.  Thank you for your time and until next time.

The End Eh!

 

ITSM vRealize Automation