Much of the truly important index data in Archiware P5 is indeed stored on tape with the data itself. \u00a0That said, ancillary information like system licensing, custom metadata and proxies, and configuration settings are often not saved. \u00a0While all of this is recoverable in the case of a true disaster, sometimes it is easier to backup the backup software itself to a different location for fast stand up in the case of a true catastrophe.<\/p>\n
Using python (2.6 for this script), here is a\u00a0basic script that automates this process for Mac and Linux P5 servers.<\/p>\n
<\/p>\n
Let’s break it down:<\/p>\n
This first section simply imports the libraries we are going to use in python and allows us to set a few simple variables, namely where P5 is installed and where we want our backups to go<\/p>\n
\r\n#!\/usr\/bin\/env \/usr\/bin\/python\r\n\r\nimport sys\r\nimport os\r\nimport commands\r\nimport subprocess\r\nimport sys\r\nimport shutil\r\nimport time\r\n\r\n#\r\n# where does your archiware install live (no trailing slash)\r\naw_path = \"\/usr\/local\/aw\"\r\n# where do you want to backup index and log files to?\r\nbackup_path = \"\/Users\/szumlins\/Desktop\/backup\"\r\n<\/pre>\nNext, we want to see if nsd (the P5 server process is running). We do this using the *nix pgrep command and looking for output. If no output is returned, we know it isn’t running and we tell variable “nr” that the software isn’t running.<\/p>\n
\r\n# check to see if nsd is already running, if it is lets throw a flag\r\nnr = 0 \r\n# our flag\r\nif(commands.getoutput('pgrep nsd') == \"\"):\r\n\tprint \"P5 is not running\"\r\n\tnr = 1\r\n<\/pre>\nNow that we know if P5 is running, we want to make sure that no jobs are active. It would be a bad idea to shut down the service in the middle of a large backup or archive!<\/p>\n
The first line checks if we should even do this step. If P5 isn’t running, then no jobs are running.
\nAfter this, we use the subprocess class to run a CLI call to nsdchat, P5’s CLI API asking for the name of all the jobs.
\nFollowing lines just clean up spaces and put all the jobs into an array so we can iterate against them in our next block.<\/p>\n
\r\n# if P5 is running, lets check and make sure no jobs are running\r\nif(nr != 1):\r\n# get all the running jobs\r\n\tjobs_str = subprocess.check_output([aw_path + \"\/bin\/nsdchat\",\"-c\",\"Job\",\"names\"])\r\n\tjobs_str.rstrip()\r\n\tjobs = jobs_str.split()\r\n<\/pre>\nNow that we know all the jobs, we only care about jobs that are actually running. Scheduled jobs in the future are registered as jobs even if they aren’t running.<\/p>\n
First, we create an iterator so we can get some metrics when we are done with our loop (i).
\nNext we enter a loop for all of the job numbers in the array we created above.
\nUsing the Job class in the P5 API, we run the status method against our job number. If that job returns “running”, we increment our iterator (i) by one.<\/p>\n
\r\n# figure out which are running and which are stopped. We only care about running jobs\r\ni = 0\r\nfor job in jobs:\r\n\tstatus = subprocess.check_output([aw_path + \"\/bin\/nsdchat\",\"-c\",\"Job\",job,\"status\"]).rstrip()\r\n\tif status == \"running\":\r\n\t\ti = i +1\r\n<\/pre>\nNow that we’ve looked at all the jobs, if all is working our (i) should be zero if it is safe to restart and should be larger if there are running jobs. We simply say that if (i) is greater than 1, send some info to the console and don’t run the shutdown process or backup process.<\/p>\n
\r\n# if there is a running job, don't stop the service\r\nif i < 0:\r\n\tprint \"There are \" + str(i) + \" job(s) running, can't shut down service!\"\r\n\texit()\r\n<\/pre>\nIf P5 is running AND there are no jobs running, we can safely stop the service. We run the stop-server command and print what it is doing to the console.<\/p>\n
\r\n#if we can shut down P5, lets do it\r\nprint \"Shutting down P5\"\r\noutput = subprocess.check_output([aw_path + \"\/stop-server\"])\r\nprint output\r\n<\/pre>\nIf we have gotten this far, we now know that our P5 service isn't running and it is safe to make copies of the online indexes.<\/p>\n
First things first, we timestamp the start process so we know how long all of this is going to take.<\/p>\n
\r\n# Lets make a backup of the files now, first we make a stamp so we know how long it took\r\ntime_start = time.time()\r\n<\/pre>\nSafety says always make a backup of your backup, so first we check to see if there is a safe previous one. If that backup directory is found, move it to a backup backup. Backup backup backup backup backup.<\/p>\n
\r\n# Next we check to see if there is already a valid previous backup. If there is, move it to keep it safe\r\nprint \"Checking if there is existing backup\"\r\nif(os.path.isdir(backup_path + \"\/aw\")):\r\n\tprint \"Existing backup found, moving old backup\"\r\nif(os.path.isdir(backup_path + \"\/aw-old\")):\r\n\tshutil.rmtree(backup_path + \"\/aw-old\")\r\n\tshutil.move(backup_path + \"\/aw\",backup_path + \"\/aw-old\")\r\n<\/pre>\nNow that we are certain that we have someplace safe to make a copy to, lets get to copying. We are copying three directories that are useful in case of a full recovery: indexes, customerconfig (server configuration), and logs.<\/p>\n
\r\n# After we make sure we can make a new copy, we copy the proper files\r\nprint \"Making backup copy\"\r\nshutil.copytree(aw_path + \"\/config\/index\",backup_path + \"\/aw\/config\/index\",symlinks=True)\r\nshutil.copytree(aw_path + \"\/config\/customerconfig\",backup_path + \"\/aw\/config\/customerconfig\",symlinks=True)\r\nshutil.copytree(aw_path + \"\/log\",backup_path + \"\/aw\/log\",symlinks=True)\r\ntime_stop = time.time()\r\n<\/pre>\nLast but not least, we calculate how long it took and start P5 back up so it can continue on its merry way.<\/p>\n
\r\n# Then we let you know how long it took\r\nprint \"Backup took \" + str(time_stop - time_start) + \" seconds\"<\/code>\r\n\r\n# Last, we start up the server\r\noutput = subprocess.check_output([aw_path + \"\/start-server\"])\r\nprint output\r\n<\/pre>\nThis script does need to be run as a root user as it starts\/stops the P5 service. Permissions on the folders it creates are default to root ownership, so you may want to noodle with the script to change permissions on your backups to make them more accessible.<\/p>\n
Best bet would be to schedule this script to run in off hours with cron or launchd as the root user.<\/p>\n
Update 11\/6\/2017<\/b><\/p>\n
I've updated the script to support Python 2.6 and 2.7 as well as to provide logging. A lot of the same methods are still in place, but the updated script is now available at https:\/\/github.com\/szumlins\/p5_self_backup<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"Much of the truly important index data in Archiware P5 is indeed stored on tape with the data itself. \u00a0That said, ancillary information like system licensing, custom metadata and proxies, and configuration settings are often not saved. \u00a0While all of this is recoverable in the case of a true disaster, sometimes it is easier to … Continue reading “Automatically backing up P5 indexes”<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[8,3],"tags":[],"class_list":["post-135","post","type-post","status-publish","format-standard","hentry","category-archiware-p5","category-general-info"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p2bwLw-2b","_links":{"self":[{"href":"https:\/\/provideotech.org\/index.php?rest_route=\/wp\/v2\/posts\/135","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/provideotech.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/provideotech.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/provideotech.org\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/provideotech.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=135"}],"version-history":[{"count":0,"href":"https:\/\/provideotech.org\/index.php?rest_route=\/wp\/v2\/posts\/135\/revisions"}],"wp:attachment":[{"href":"https:\/\/provideotech.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=135"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/provideotech.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=135"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/provideotech.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=135"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}