Updating SSL Certificates for CloudFront Distributions

Recently we had to upgrade our CloudFront distribution because our SSL certificates were set to expire.  Because we use CloudFront as a CDN for our images and for our Rails assets cache and we serve all of our assets using SSL, we had to update the SSL certificate that our CloudFront distro uses:

edit-cloudfront-distribution-ssl-settings

Since we are using a custom domain CloudFront will need to serve our SSL certificate for all SSL requests – otherwise all of our cached assets and images will fail with an ERR_INSECURE_RESPONSE for all requests to our CloudFront distribution.  Note that the screen shot above shows the part of the CloudFront distribution UI where you have to pick the SSL certificate for your distribution.

This is the important part: if you have an existing SSL certificate, Amazon will not let you simply overwrite the existing certificate.  You will have to upload a new certificate using the aws cli command line tools, and then switch the active certificate in this UI or using the command line.  Note that this will trigger a redeploy of your distribution which can take a long time.

The command line for listing your certs in iam:

aws-list-cloudfront-certificatesCloudfront expects the certificates it stores to live in the /cloudfront/cert directory (under “Path” above).  For more info on setting this stuff up from scratch, check the AWS CloudFront site for the latest incantations.

The command to upload a new certificate to IAM:

aws iam upload-server-certificate 
--server-certificate-name BlinkStarCertificate2016 
--certificate-body file://./certs/star_blinkinc_com.crt 
--private-key file://../BlinkIncCerts/star_blinkinc_com.key 
--certificate-chain file://star_blinkinc_com_cf.pem  
--path /cloudfront/cert/

Note that you will not be able to overwrite your old certificate name unless you delete it first. We chose to go with creating a new cert name and making sure that works, then deleting the old cert.  Note that the –certificate-body option must point the .crt file you get from your issuer.  The –private-key file is the private key you used to sign your cert. The –certificate-chain option specifies a .pem file that only contains the certificate chain for the root and any intermediate certificates but not the actual site certificate.  If you this command complains that “A client error (MalformedCertificate) occurred when calling the UploadServerCertificate operation: Unable to validate certificate chain. The certificate chain must start with the immediate signing certificate, followed by any intermediaries in order. The index within the chain of the invalid certificate is: 1” you probably have your site cert in the .pem file.  Remove it and you are good to go!

Oh, also note that Amazon apparently still thinks Java based command line tools are cool somehow.  Ugh.  Hence the file:// URL style syntax in the command line.  Ugly.  Awful.

Result Values From a JSX Script

When writing Generator plugins for Photoshop you often have to call snippets of JSX code which do your actual work in PS.  This integration is a little tricky to work with, especially when dealing with finding out what happened in your JSX script when it executed in PS!

Basically, each JSX script you call will be executed in the JSX interpreter, not in the node container your generator is running in.  Adobe provides only one return value that your JSX can run, a variable called ‘result’.  If you carefully construct your script to catch all exceptions during execution you will be able to set this result variable to detect issues that might occur during execution.

You have to remember though:  the result variable can only be the *last* thing you set in your JSX code, you cannot execute anything after doing something like this:

try {

    ...

    function blinkImageProcess() {

        runProcess();

        return 'OK';
    }

    // JSX interpreter looks for the 'result' var and returns it to the caller

    result = blinkImageProcess();
} catch (e) {
    result = e.message;
}

Adding code after the ‘result = …’ statement above will cause the JSX interpreter to return ‘undefined’ for ‘result’! Major fail.

Photoshop Generator Node Plugins and Error Handling

At Blink Inc we have been writing Photoshop plugins that use their new-ish ‘Generator‘ Node-based plugin technology.  We use some of these plugins to process images in real-time from our photo studio.

The Generator technology makes writing PS plugins a lot more pleasant than using the other alternatives Adobe has developed in the past however there are some gotchas involved around the requirement to use the older JSX code to invoke PS actions from Node. Hopefully Adobe continues to develop and support their node framework for plugins so this gets more tightly integrated.

The main problem with the way plugins work has to do with a common Javascript issue – uncaught exceptions.  It turns out that the code you pass to Photoshop using

evaluateJSXString(...)

is not wrapped in a try-catch block by default! This *will* cause you maddening debugging sessions where your code will mysteriously not function, or will appear to not return from a call.

In addition, exceptions from the JSX interpreter are *not* passed back from the

evaluateJSXString(...)

call, so we strongly recommend you make sure to wrap your JSX code in a try / catch block. All of the JSX code we pass to the JSX interpreter is wrapped in:

 try {

   // Your JSX code goes here

   result = callYourProcessingCodeWithAFunction();
 } catch (e) {
   result = e;
 }

Since we do a lot of file manipulation in PS this can be used to at least diagnose what went wrong during testing or in production via logging.  Since you can get a result object back from the JSX interpreter, you can use this to return the result of your operation (usually a string if success) or return the error object you get if an exception gets thrown.

Retrying Image Transfers with the Canon 1Dx

When there are network or camera glitches during image transfers from the Canon 1Dx images will need to be manually transferred to the Blink FTP servers for processing, preferably during a photo shoot or at least as soon as the image transfer errors are noticed.

STEP 1:

Image 1
Canon 1Dx PLAY2 menu

You are going to walk through 6 more steps to try and transfer images for processing. First thing to do is select the Image transfer menu in the PLAY 2 top-level menu on the 1Dx.

STEP 2:

Image 2
Select the ‘Image sel./transfer’ menu

Select ‘Image sel./transfer’ – this will let you select the images that you will be transferring via FTP.

STEP 3:

Image 3

This screen shows how many images failed transfer during the 1Dx’s attempts to transfer them. This screen also allows you to pick the images to try transferring them again.  If there are images that failed transfer, then you have to select the ‘All image’ button on this screen, then follow the next step.  It is also possible to manually select images by folder (Sel. <folder icon> button) or individually (Sel. Image button).  Generally we do not want to do this at Blink unless we need to reprocess the images from an entire shoot or several shoots.

STEP 4:

Image 4

This screen lets you pick what images to transfer in the next step.  If the camera listed images that failed transfer in the previous screen, then you will want to select ‘Card images failed transfer’ here.  The other two options will let you select images that were never transferred (this does not apply to Blink since we tell the camera to transfer every frame shot in the camera) and to clear the transfer history stored on the current CF card. The 1Dx stores a record of transferred images on the CF card – this is not super well documented by Canon. We recommend not clearing the CF card transfer history unless you are 100% certain that all images that were meant to be transferred actually get sent from the camera.

STEP 5:

Image 5

Once you have selected the images to transfer, you will get sent back to the screen in Step 3.  Notice that it now says there is one image in the transfer queue.  Now you can select the ‘FTP transfer’ button down below.  This will take you to the next step…

STEP 6:

Image 6

Click OK here to start the transfer.  Clicking Cancel will take you back to the previous screen.

STEP 7:

Image 7

The 1Dx is now transferring images.  This screen shows the status of the transfer.  This screen will list each image as it is sent from the camera.

STEP 8:

Image 8

After the images you selected are transferred, you will end up back at the image selection screen.  In this example, the one image we selected was successfully transferred, but if there were errors during the transfer the camera would show how many images failed to transfer on this screen.  We recommend that if there are errors in the transfer at this point that you note the actual error that happened by going to the error description (follow the steps in the post about Troubleshooting FTP Transfers with the Canon 1Dx ). Unfortunately the errors that occurred are not shown on this screen and it takes quite a bit of navigation to track down the actual error that happened!

Troubleshooting FTP Transfers with the Canon 1Dx

Figuring out what went wrong with wireless image transfers from the Canon pro cameras can be really difficult; at least in our experience with the Canon 1Dx with the WFT-E8A wireless transmitter.

Canon gives us a status light on the back of the camera (lower left corner, really tiny). When this status light is blinking red, something went wrong.  This is OK when there are network problems because the light will keep blinking red and no images will transfer at all.  When a shoot is going on, there is no time to catch the light blinking red to indicate an image transfer went wrong.  Since we transfer images more or less continuously during a shoot and there are usually hundreds of images shot in a short amount of time, it is really hard to catch the light blinking while also shooting!

For us the most common cause of failed image transfers is failed writes to a CF card.  We seem to burn through CF cards very quickly.  This could be because the Blink cameras shoot images very rapidly over a short period of time, or just the large number of writes we are doing to the CF card’s flash memory.  Either way, this is a very tricky problem to deal with for our photographers because images will seem to mysteriously not show up on our processing server.

The 1Dx makes it somewhat difficult to troubleshoot network errors – the error page is buried under the ‘Communication settings’ submenu so it is not super easy to find.  See below for the three steps it takes to get to the ‘Error description’ menu!

Toolbar 3Toolbar 4Toolbar 5

If images are not transferring this is the first place to look for troubleshooting clues.  If there was an error the camera will place an error message here, which can help to troubleshoot the problem.  Again, if there are transfer issues, look here first. Unfortunately, the error message on this page gets cleared after you attempt to transfer images with errors, which is why it helps to get a cell phone shot of this screen before attempting additional transfer troubleshooting steps.

For a step-by-step on transferring images with errors (for example if you determine that there was a temporary network error here) see:  Retrying Image Transfers with the Canon 1Dx

Check to see if the camera is connected to the wireless network:

Check to see if the camera has a valid IP address:

Check to see if the camera can connect to FTP:

Double-check your FTP username and password:

Make sure FTP is running:

Make sure FTP allows connections from your camera:

 

Setting up FTP Transfers with a WFT-E8A on a Canon EOS 1Dx

Setting up FTP for the Canon EOS 1Dx camera

At at the Blink Inc photo studios our primary requirement has been to get images from a full-frame camera with high-quality lenses to our photo processing software in Amazon EC2 and then to Amazon S3 in under 7 seconds so that customers can see their images during their shoot.

Since we are using Canon pro gear we have had to figure out how to get Canon’s cameras to play along with our processing software using FTP. Judging by the amount of frustration vented in some of the forums about this there is a lot of frustration our there about FTP configuration.  Not surprising – FTP has many different options there are quite a few ways to screw this up.  Also Canon has gone with the option to provide every FTP option possible in their user interface on the camera, so a person that is not intimate with FTP is going to have to do a lot of research just to get a basic setup going to transfer their images this way.

Hardware

The Canon 1Dx can be configured to use the WFT-E8A wireless transmitter.  This setup is actually really nice for a lot of reasons – our previous cameras were 5D Mark IIs using WFT-E4 IIA wireless transmitters and these did not support 802.11 ac.  The performance of the newer transmitter is noticeably better (we have benchmarks, if anyone is interested we are happy to share in another blog post).  Also, the attachment to the camera itself is much more secure, since there is a big thumbscrew that allows for a very secure attachment point to the camera body.  I also think that the position of the transmitter is ideal because it is out of the way of the photographer.

wft-e8a-3q-675x450
Canon WFT-E8 Wireless Transmitter

The software configuration in the camera however is not that much better in the 1Dx – in fact it I am documenting it here because it is so hard to figure out all the options that need to be set, and where the settings are! There are also a lot of options as far as what sorts of images the camera will send to the FTP server when the shutter is triggered.

At Blink, we deal only with low-compression JPEG files.  For some photographers RAW will be preferred however this will add an enormous transfer time penalty since RAW files are huge.  More on this later.

FTP Server Setup

Our FTP servers live in Amazon EC2 – we use VSFTP and there are FTP servers in multiple regions for redundancy.  Amazon Route 53 provides us with a way to effectively load balance upload requests between FTP servers from multiple studios (since we now have 4 and more on the way).

Our FTP servers use passive FTP since we are behind a NAT firewall and there is no way to NAT ports through Route 53. Additionally we have DNS round robin set up in Route 53 – which means the cameras have to be configured to use DNS.  Thankfully Canon added this to the WFT-E8A firmware – older versions of Canon hardware did not have a way to configure DNS!  If your cameras are sending images to an FTP server on the internet, there is a very good chance that you will have to use passive mode FTP.

Camera Setup

Screw the WFT-E8A transmitter to the camera body. The camera should have a CF card inserted otherwise the camera will complain when you try to capture images. The CF card needs to be formatted if it is not already.

Set the camera up to join a wireless network.  At Blink all images are shot hand-held so the camera is not tethered to an ethernet plug.

Set the DNS settings for the wireless network.

Set the FTP server settings.

Make sure the camera settings have the color space settings you want.  The 1Dx enforces an image name format based on the colorspace of the images.  At Blink we use the SRGB color space.

For Blink, we want image names that are formatted like:

IMG_[CAMERA ASSIGNED NUMBER].IMG

For any other colorspace the 1Dx will not let you set this format!

Lastly, you will want to tell the camera how to transfer images once they are shot.  By default the 1Dx firmware is set to not transfer images on capture.  At Blink we want automatic transfers, so the last step for us here is to set the camera to automatic transfer.

Once this is all set up, you can shoot some test frames to make sure images are being sent. Unfortunately if something goes wrong, Canon does not make it super easy to figure out what has gone wrong.  This is a topic for yet another blog post: Troubleshooting 1Dx FTP Transfer Issues.