Cloud Foundry: Understanding the Core Components
Explore the key components you'll need to build an entire Cloud Foundry architecture.In a recent post, I spoke about some of Cloud Foundry's main...Learn More
Pivotal Cloud Foundry (PCF) is an open source platform based on Cloud Foundry and offered as a collaboration between Pivotal, EMC, and GE. Pivotal Cloud Foundry runs on almost all popular cloud infrastructures, including VMWare, AWS, and OpenStack. PCF as a platform is dynamic, developer friendly, and features full-lifecycle support.
Organizations implementing Pivotal Cloud Foundry as their cloud platform free themselves from managing application infrastructure. When integrating freely available third-party tools and services, they can also achieve high-availability, auto-scaling, dynamic routing, multi-lingual support, and log analysis.
Pivotal Cloud Foundry performs exceedingly well when intelligently designed and maintained, but there are still some time-consuming tasks that demand an admin’s attention. One such operational task is ensuring that installation settings and essential internal databases are regularly backed up. Pivotal recommends that you back up your installation settings by exporting them at regular intervals (weekly, bi-weekly, monthly, etc). We’re going to discuss designing an effective and reliable back up process…and how to apply an archive when you need to restore your installation.
Note: According to Pivotal Cloud Foundry documentation, exporting your installation only backs up your installation settings. It does not back up your VMs or any external MySQL databases that you might have configured on the Ops Manager Director Config page.
Before jumping in, it’s a good idea to make sure that you’ve covered all the prerequisites you’ll need to make Pivotal Cloud Foundry happy. You’ll need:
Backing up a Pivotal installation is critical for the operation and availability of your Pivotal Cloud Foundry data center. Backing up Pivotal Cloud Foundry data centers is like creating restore points on a Windows machine. In the event of a crash or the failure of an upgrade process, you can restore your back up settings to fall back to an earlier, functional image. Here’s what you’ll need to do:
To make sure that your system is ready for the process, there are some important details that will need taking care of in the pre-backup stage:
Pivotal Cloud Foundry’s Cloud Controller Database maintains a database with records of orgs, spaces, apps, services, service instances, user roles, etc. Backing up this database is critical if you want to protect your existing settings (and you DO want to protect your existing settings).
$bosh target <IP_OF_YOUR_OPS_MANAGER_DIRECTOR> $bosh login
Your username: director Enter password: Logged in as `director'
$bosh deployments >> /pcf-backup/deployments_09_20_2015.txt
$bosh download manifest DEPLOYMENT-NAME LOCAL-SAVE-NAME $bosh download manifest cf-1234xyzabcd1234 cf-backup-09_20_2015.yml
$bosh deployment cf-backup-09_20_2015.yml
$bosh vms cf-1234xyzabcd1234
$bosh -d cf-backup-09_20_2015.yml stop cloud_controller-partition-cdabcd1234b253f40 $bosh -d cf-backup-09_20_2015.yml stop cloud_controller_worker-partition- cdabcd1234b253f40
ccdb: address: 184.108.40.206 port: 2544 db_scheme: postgres
vm Credentials vcap / xyz1234567989pqr
$find /var/vcap | grep 'bin/psql'
Your output should look something like this:
$/var/vcap/data/packages/postgres/<random-string>/bin/pg_dump -h 220.127.116.11 -U admin -p 2544 ccdb > ccdb_09_20_2015.sql
#scp email@example.com:/home/vcap/ccdb_09_20_2015.sql /pcf-backup
This will complete the CCDB backup process.
vm Credentials vcap / xxxxxxxxxxxx Credentials root / xxxxxxxxxxxxxxxxxx
$find /var/vcap | grep 'bin/psql'
#/var/vcap/data/packages/postgres/<random-string>/bin/pg_dump -h 18.104.22.168 -U root -p 2544 uaa > uaa_09_20_2015.sql
# scp firstname.lastname@example.org:/home/vcap/uaa_09_20_2015.sql /pcf-backup
This completes the UAADB backup process.
The Console Database is referred to as the Apps Manager Database in Elastic Runtime 1.5.
Vm Credentials vcap / xxxxxxxxxxxxx Credentials root / xxxxxxxxxxxxxxxxxxx
$find /var/vcap | grep 'bin/psql' $/var/vcap/data/packages/postgres/<random-string>/bin/pg_dump -h 22.214.171.124 -U root -p 2544 console > console_09_20_2015.sql
$scp email@example.com:/home/vcap/console_09_20_2015.sql /pcf-backup
This completes the Console Database backup process.
$tar cz shared > nfs_09_20_2015.tar.gz
$scp firstname.lastname@example.org:/var/vcap/store/nfs_09_20_2015.tar.gz /pcf-backup
This completes the NFS Server backup process.
Backup your MySQL Database:
$bosh download manifest p-mysql-abcd1234f2ad3752 mysql_09_20_2015.yml
$mysqldump -u root -p -h 126.96.36.199 --all-databases > user_databases_09_20_2015.sql
This completes the MySQL DB backup process.
$bosh -d cf-backup-09_20_2015.yml start cloud_controller-partition-cdabcd1234b253f40 $bosh -d cf-backup-09_20_2015.yml start cloud_controller_worker-partition- cdabcd1234b253f40
Restoring a Pivotal Cloud Foundry deployment requires that you reinstall your installation settings restoration and key system databases. Or, in other words, everything we backed up in the previous operations. You’ll need to follow these steps:
We’ll use the UAADB as an example. The rest will follow the same process.
$ bosh stop <uaa job>
$ssh vcap@[uaadb vm ip]
$/var/vcap/data/packages/ /postgres/<random-string> /bin/psql -U vcap -p 2544 uaa
drop schema public cascade; create schema public;
$scp uaa.sql vcap@[uaadb vm IP]: #UAADB server $/var/vcap/data/packages/postgres/<random-string>/bin/psql -U vcap -p 2544 uaa < uaa.sql
$bosh start <uaa job>
A Pivotal Cloud Foundry backup process can be scheduled (and scripted) to create restore points for your installation. You could also use the settings file backups to launch a new installation in a different availability zone or even on a different platform. PCF has provided excellent documentation on both the backup and restore process.
Thoughts? Add your comments below.
Being able to architect your own isolated segment of AWS is a simple process using VPCs; understanding how to architect its related networking components and connectivity architecture is key to making it a powerful service.Many services within Amazon Web Services (AWS) require you t...
AWS is renowned for the rate at which it reinvents, revolutionizes, and meets customer demands and expectations through its continuous cycle of feature and service updates. With hundreds of updates a month, it can be difficult to stay on top of all the changes made available. Here ...
Amazon Web Services (AWS) offers three different ways to pay for EC2 Instances: On-Demand, Reserved Instances, and Spot Instances. This article will focus on effective strategies for purchasing Reserved Instances. While most of the major cloud platforms offer pre-pay and reservation dis...
If you’re building applications on the AWS cloud or looking to get started in cloud computing, certification is a way to build deep knowledge in key services unique to the AWS platform. AWS currently offers 11 certifications that cover major cloud roles including Solutions Architect, De...
The AWS Solutions Architect - Associate Certification (or Sol Arch Associate for short) offers some clear benefits: Increases marketability to employers Provides solid credentials in a growing industry (with projected growth of as much as 70 percent in five years) Market anal...
Moving data to the cloud is one of the cornerstones of any cloud migration. Apache NiFi is an open source tool that enables you to easily move and process data using a graphical user interface (GUI). In this blog post, we will examine a simple way to move data to the cloud using NiFi c...
Amazon DynamoDB is a managed NoSQL service with strong consistency and predictable performance that shields users from the complexities of manual setup.Whether or not you've actually used a NoSQL data store yourself, it's probably a good idea to make sure you fully understand the key ...
As companies increasingly shift workloads to the public cloud, cloud computing has moved from a nice-to-have to a core competency in the enterprise. This shift requires a new set of skills to design, deploy, and manage applications in cloud computing.As the market leader and most ma...
Learn how Aviatrix’s intelligent orchestration and control eliminates unwanted tradeoffs encountered when deploying Palo Alto Networks VM-Series Firewalls with AWS Transit Gateway.Deploying any next generation firewall in a public cloud environment is challenging, not because of the f...
Use AWS Config the Right Way for Successful ComplianceIt’s well-known that AWS Config is a powerful service for monitoring all changes across your resources. As AWS Config has constantly evolved and improved over the years, it has transformed into a true powerhouse for monitoring your...
Cloud Academy is a proud sponsor of the 2019 AWS Summits in Atlanta, London, and Chicago. We hope you plan to attend these free events that bring the cloud computing community together to connect, collaborate, and learn about AWS. These events are all about learning. You can learn how t...
The AWS cloud platform has made it easier than ever to be flexible, efficient, and cost-effective. However, monitoring your AWS infrastructure is the key to getting all of these benefits. Realizing these benefits requires that you follow AWS best practices which constantly change as AWS...