In a natural extension of the PresSTORE/CatDV work I have been doing, I have written a more unified installer and configuration tool for the scripts on Mac OS X. The project is called Castor and it has its own page now here on my blog. Check the page for updates to the scripts or if you have any questions.
Hey Mike, this is generally working great!
I am having an issue though when submitting files to PresStore that have parenthesis in the filename or any parent directory names. The log says that PresStore did not generate an archive handle and the path gets logged in aw-queue-err.log. I’m wondering if in the “run archive method” section of aw-queue.pl, around line 195-217 there’s a loop that starts with while (). In that section there’s a line:
@handles[$i] = `$nsdchat ArchiveSelection $archive_selection addentry {$_}`;
Wondering if the {$_} should be enclosed in double quotes. Or is there another way to escape shell metacharacters? Seems like that’s probably the issue.
Jason, thanks for the feedback. Always appreciated to get some. The problem you are running into has to do with special characters being interpreted without escapes by the shell command in that archive loop. I have updated the aw-queue.pl script in the git repository with some common special character escapes. I haven’t had a chance to build it into the installer, but grabbing the latest version from http://github.com/szumlins/Castor should get you going.
Thanks for the quick reply Mike. I’ll check it out. Another great thing to include would be a way to handle the paths that do not generate archive handles – maybe add them to an error-queue.txt or something, so that there’s a record of files that did not archive.
I have seen another issue related to our particular installation – Our root password on our PresStore server contains six characters that are considered special to the shell. One is the @ symbol. No matter what I try, I am unable to log into nsdchat through sockets when that @ is there. I tried changing the password to simple text and numbers and it worked fine. I also tried adding back in the special characters one at a time, and escaping them with a preceding backslash. This also worked. However, when I tried to add the @ symbol back in, it didn’t work, even when escaped. nsdchat just sits there. To remedy this, I changed the line in aw-queue.pl that defines the $nsdchat to use awsock – I just made it call nsdchat in “/usr/local/aw/bin/nsdchat -c”, and it worked fine. So it would appear that in addition to a colon, you cannot have an @ symbol in your password if you intend to use sockets to connect to PresStore.
Also, you may be interested to know that we have Castor working on our system, and our asset management functions are split into three separate servers. One is running Worker Node and Episode Engine, one is running CatDV server and hosting proxy files, and the other is running PresStore. Normally I probably would have installed nsdchat on the Worker Node server and used sockets to command PresStore, but with the password issue I wanted to try keeping things separate. All of our servers and local administrator accounts have the same admin password so I wanted to try something different. I was able to make this work by putting the queue files on our Xsan in a directory to which both the Worker Node machine and PresStore server have access. I had to change the launchd plist files to point to those queue files. I also had to install catdv2aw.pl on my Worker Node server, but as you indicated in your help file, that script isn’t 100% necessary as it only adds the media path to the queue file.
The next enhancement I’m working on is a way to get the job number from PresStore back into CatDV so that there’s an archive history record of the job. Also, with that job number, we should be able to use nsdchat to get a barcode number of the tape(s) used in that job. That would be helpful. I see that you have aw-queue.log reporting the job number after the archive run, so I’m sure I can make this happen. I suppose the queue file would list the job number on the first line, the barcode(s) on the second, and the remaining lines would be paths to files that were archived in that job. I’ll let you know how this goes, and if you have any other ideas let me know.
Thanks and good work!
Sorry to bother you Mike, I’m not seeing the update on github. All I see on aw-queue.pl is:
aw-queue.pl 3 months ago Modified paths to no longer be relative. Fixed two typos. [szumlins]
Jason, apologies. I wrote the changes on site for another user, copied it to my evernote notes, and then never committed to github. Check it again, it should have the up-to-date version now with new commit language.
To return the tape information for each asset, you are going to need to look up the archive entry name for each archive handle after the job has completed. Right now Castor doesn’t store that information anywhere to be looked up after the script, but you could easily write it to a temp file and then on a successful job status complete, poll the archive index for those handles and recover the location. You would have to format this in XML and hand it back to worker in a watch folder as well.
This will return volume information where $handle is the Archive handle generated during write
nsdchat -c ArchiveEntry $handle volume
This will return label information for the volume, where $volumename is returned from the above command
nsdchat -c Volume $volumename label
This will return the tape barcode for the volume, where $volumename is returned from the above command
nsdchat -c Volume $volumename barcode
Hope this helps. If you come up with anything of interest and it is working well for you, let me know and I will commit it to the public git repository
Thanks Mike. That’s very helpful. I had toyed around with the PresStore CLI a bit and came up with something similar, so looks like we’re on the same page. I had thought of using Worker’s CLI to feed the info back into CatDV, but generating an XML would probably work even better. Somehow in my installation, the barcode that’s returned by the Volume barcode command isn’t always the right number, it’s some weird 4-digit number instead of the barcode on the tape. Doesn’t seem to be a rhyme or reason – some are correct, others have the 4-digit number. But I’ve submitted a support ticket to AW so we’ll see!
I checked out your mod to aw-queue.pl, that looks good. I found another way of doing it in one line:
$_ =~ s/([\&;\`’\\\|”*?~^\(\)\[\]\{\}\$\n\r])/\\$1/g;
Cheers, and thanks for the info! I’ll keep you posted on what I come up with.