Heroku vs Replicate
I’ve recently found that the
heroku run command can randomly truncate output so your milage on this may vary, especially with larger sets of data. GitHub Issue here: https://github.com/heroku/heroku/issues/674
While trying to use the Replicate gem with Heroku, I ran into a strange issue. Namely, Heroku didn’t like the bytestream that Ruby’s Marshal class was producing:
1 2 3 4 5 6
I did find a workaround though. The
base64 binary is present on Heroku VMs so you can pipe replicate’s output through that to get data that Heroku can handle. However, there are a couple caveats here:
Heroku squashes STDERR into STDOUT which means you need to filter out STDERR (via
2>/dev/null) before sending the data back to the client.
You also need to do a little manual work on the data you get back to filter out the lines Heroku addes to the output.
Decode the base64 file on your end before passing it your local replicate instance.
Here is the flow that works for me:
file.dump and remove any of the “…attached to terminal” lines Heroku adds. In the following case, just the first line.
1 2 3
Decode the base64 file back into a Marshal friendly file and load it into Replicate:
1 2 3 4 5
Hope that helps anyone working with Replicate on Heroku.