Migrating Caprover to Portainer

Before I fully understood Docker Services and Swarm, I needed an easy way to host containers for personal and business use. Caprover fit that solution perfectly. I was able to spin up premade images / stacks, and manage them with basic functionality. But these days I need finer control, and that’s where Portainer comes in. Portainer is perfect if all you need is a UI to upload Docker Compose files and spin up services there. It also offeres easy configuration and management of other Docker features such as Networks, Volumes, and Nodes.

Caprover runs off of docker containers, just like Portainer. So to migrate to Portainer, I just had to install Portainer, shut down the Caprover containers, and “take over” the containers. Taking over the containers consists of rebuilding the images in a docker compose file, and using the same volumes as before for data persistence.

The overall steps look like this:

1. Configure your firewall to allow Portainer.
2. Install and run the Portainer docker image.
3. Access Portainer over your web browser and complete the initial configuration steps.
4. Create your Traefik stack to replace Caprover’s NGinx Proxy. Stop the stack immedietly as Caprover’s is still running and it won’t be able to bind to the ports.
5. Rebuild your services in Docker Compose.
6. Copy the data from your old Caprover volumes to your new volumes. (More on this later)
7. Shutdown the Caprover services
8. Start your new stacks. (including Traefik)

Migrating the volumes is probably the trickiest step. Caprover doesn’t follow the normal Docker naming scheme of <STACK NAME>_<VOLUME NAME>. Instead it’s something like captain--<CONTAINER NAME>--<VOLUME NAME>.

I was hesitant to rename the volumes as it may mess with other Docker processes, so here’s what I did. I created the new Volume, and copied the data from the captain volume to the new one. Make sure to use rsync to preserve permission and group data for each file. Overall command looked something like this: rsync -raz _data <NEW VOLUME> _data will be copied under the new volume.

All in all that should be it.

Image Manipulation with Rails

For one of my clients I was tasked with generating ID cards in a web app that runs on Ruby on Rails. Initially I wasn’t sure how I was going to tackle this. Rails isn’t well known for image manipulation. However, there is a little bit of image manipution built into rails, with Active Storage varients. From what the Rails docs say, varients use RMagick on the backend, and RMagick is simply a wrapper for ImageMagick, a popular package for processing and manipulating images. The Rails docs indicated that vips, a lighter and faster image processor, will become default soon. But for this example I will use RMagick.

I wanted to write this post because while the documentation for RMagick is very comprehensive, there’s very little in the way of examples. So hopefully this will help someone else with a similar application. This will cover two things. Adding text to existing images, and adding an image on top of another image. Essentially using an ID card template and adding the user’s name, downloading a cropped and compressed profile picture from active storage, adding said picture to the ID card template, and uploading the new image back to active storage.

This code can be ran in both a controller and job. If possible I would recommend putting this in a job, since it can take a while on slower hardware, but it really depends on the control flow of your app. In my case, I needed to display the manipulated image to the user and they needed to immedietly download it, so I stuck it in a controller, and pretty much just told the user that they will have to wait a second or two for the image to be processed.

Loading your image

First thing we need is an object that represents out base image that we can manipulate with RMagick. That object will be an ImageList object. Here’s what that code looks like:

new_card = ImageList.new(PATH_TO_IMAGE) do
	self.format = 'jpeg'
end

An ImageList is actually an array of images. If you’re just working with one image and adding text to it, you can use the regular image object. But this tutorial is using an ID card as an example, so we’ll be working with multiple images. If you’re not sure, just leave it as ImageList

The format can be whatever you want, but I’d recommend something common like jpeg. After we’ve manipulated our image, it will be saved in this format. For compatibility reasons, I would also make sure your template image matches this format. When I started writing this code I used .tif templates because I figured “Hey higher quality is better and it’s getting processed anyways”. RMagick does not handle tif images well, so I converted them to jpegs.

Adding Text to Images

With RMagick, adding text is pretty straight forward. Take a look at this code below.

Draw.new.annotate(new_card, text_box_width, text_box_height, text_left_x, y, text) do
  self.gravity = Magick::CenterGravity
  self.font = font_path
  self.fill = text_color
  self.pointsize = text_size
end

Let me explain what each variable represents.

new_card is the the ImageList object, holding our template image.

text_box_width and text_box_height. These are two integer values that represent the box you want your text to go in. When you set the gravity of the text, aka how it will align, it will use these values to determine that. So if you have a width of 200, and you center the text, it will center 100 pixels off of the left most pixel.

text_left_x is the left most pixel of your text box. y is the y value. And text is a string representing the text you want to write to the image.

RMagick does not automatically wrap text if it exceeds the text_box_width. Instead it’ll just overlap. So if you expect to wrap text, you’ll need to write your own function to do so. In my code I wrote functions to both wrap and shrink text. So the code above was wrapped in a for each function that looks like this.

lines.each do |text, y, text_size|
	# shadow is drawn first so it's on the bottom
	Draw.new.annotate(new_card, text_box_width, text_box_height, text_left_x + shadow_size, y + shadow_size, text) do
	  self.gravity = Magick::CenterGravity
	  self.font = font_path
	  self.fill = shadow_color
	  self.pointsize = text_size
	end

	Draw.new.annotate(new_card, text_box_width, text_box_height, text_left_x, y, text) do
	  self.gravity = Magick::CenterGravity
	  self.font = font_path
	  self.fill = text_color
	  self.pointsize = text_size
	end
end

Lines is an array of arrays. Each sub array contains 3 values. text, y, and text_size. This code also includes text shadowing. I simply right the shadow text first, offsetted by a preset integer value (I used 1), then I wrote proper text on top of that text. So in my case the shadow was just text written in black, and then text written in white on top of that.

The Draw object has a lot of options. The 4 main ones are the ones I used. If you need other options check out the documentation here.

self.gravity needs to be a Magick Gravity Constant.

self.font needs to be a string represeting a path to the font file. So I used something like this. Rails.root.join('FOLDER', 'FILE.ttf').to_s.

self.fill is the color of the text. It is stored via hex values, so should look like this: text_color = '#FFFFFF'.

self.pointsize is just the text size. It’s an integer.

Adding an image on top of another image

So we have our card template new_card with a bit of text on it. Now we want to add someone’s profile picture to the card. Assuming you already have a profile picture ready to go that’s cropped down to a set size, here’s how you add that image ontop of our edited template.

profile_image = Image.from_blob() # you can source it from anywhere. from_blob is how you would get it from ActiveStorage

new_card.push profile_image # Remember new_card is an ImageList, meaning it can contain multiple images

profile_image.page = Rectangle.new(target_width, target_height, pos_x, pos_y) # this sets the position of the profile image on the template. target width and height should match the size of your cropped profile picture, and pos x and y is the position on the template image.

new_card_path = Tempfile.new("id_card").path # this is a temp file we can use to store the image on the file system before uploading it back to active storage

new_card.format = 'jpeg' # make sure it's saved as a jpeg
new_card.flatten_images.write(new_card_path) # then combine all the images, and save it to that temp file

# then if you want to store it with ActiveStorage, do this.
your_model.id_card_image.attach(io: File.open(new_card_path), filename: "something.jpg") # your_model being the model you're attaching it too, and id_card_image being the ActiveStorage association in that model

Honestly that’s about it. The hardest parts for me when figuring all of this out is writing a way to automatically wrap lines for large paragraphs of text, and facial recognition to automatically crop a profile image with the face in the center of the photo. These problems are not the focus of the article, and ended up being somewhat niche for my requirements.

If you have any questions about this feel free to shoot me an email.

Host a Static Website on Firebase

If you’ve ever needed to host a static website for free, Firebase is a good start. It’s simple to use, and everything on it is behind a CDN, so you’re getting excellent load times pretty much anywhere in the world. This guide will be broken up into several large steps, with a few sub-steps. If you have any questions, feel free to email me.

1. Setup your domain

Every website needs a domain. I’d recommend setting up every domain you have on Cloudflare. It’s free, it caches your content around the world similar to what firebase does, and it offers a lot of security features.

Connecting a domain to Cloudflare is pretty straight forward, and not the focus of this guide. If you want a step-by-step guide for that, Cloudflare has an official guide here. Once it’s connected, we’ll need to setup the Firebase project.

2. Setup Firebase Project

Navigate to Firebase’s website, login with your google account, and click on “Go to Console”. From here you’ll click on “+ Add Project”. Then there are 3 steps to create a new project.

1. Project Name: This can be anything. I’d suggest keeping it short and to the point.
2. Google Analytics: I’d recommend turning this off, or keep it on. It doesn’t really make a difference. Cloudflare does have their own analytics if you’re interested in something a little more privacy focused.
3. Google Analytics Optional: If you enabled Google Analytics in the previous step, here you’ll select an analytics account.

Now that the Firebase project is created, we need some code for it.

3. Create a git repo

With your preffered git provider (I recommend GitHub for later firebase integration), we need to create a repo for your website. If you already have one, you can skip this step. So create a git repo, and if during the setup you’re offered to commit a readme and license to your new repo, do that.

Then clone your repo onto your computer. At this point, there’s only one branch in your repo and it’s probably called “main” or “master”. Using your preffered git client, create a new branch from that commit and call it something like “release”. We’ll have at least 2 branches with this project. A master branch for development, and a release branch. The release branch will be what will be synced with your live website. With a GitHub repo, Firebase can auto-deploy your release branch as soon as it is committed to.

4. Setup Firebase in Project

In your terminal, you’ll want to navigate to your git repo. If this is the first time using Firebase, which if you’re reading this it probably is, you’ll want to install the Firebase CLI tools. With Node installed on your machine, you simply need to run npm install -g firebase-tools to install the Firebase CLI. And if you haven’t already, run firebase login to login to your Firebase account.

After that, it’s time to initalize our Firebase project. Simply run firebase init and follow the on screen instructions. We’ll want to select hosting. Firebase has a lot of options, that you can enable later to add functionality to your website beyond serving static code. But for that purpose, we’re going to select the first “Hosting” option.

Then we’re going to select “Use an existing project”.

And then we simply choose our already created Firebase project in the list.

Now we’re going to setup the hosting aspect of our project. We’ll want to keep public as our public directory, because that makes sense. But we’re going to say no to the single page app. I’ll explain this later, but if you want your website to have multiple pages, say no to this. And we’re going to want to enable automtic builds and deploys with GitHub. Saying yes to that option will automatically connect Firebase to your GitHub account in a similar way Firebase logged you into their CLI.

First it’ll ask us for the repository. The format for this is GITHUB_USERNAME/REPO_NAME. As you can see for me it was dalenw/test-firebase. We’ll want to say “No” to the build script. If you were using a framework like Jekyll or Vue to generate static content, you could auto-build with the command you put in. But this guide isn’t about that; this is to serve raw HTML.

Next we’ll want to setup auto-deployment. So give the “Yes” option like I did and type in the name of your release branch that we created earlier. What this means is that everytime the release branch is updated, usually through a pull request, it will tell Firebase to download the release branch and host it. So with this you can edit your master branch as much as you want, and Firebase won’t host your changes until it’s in the release branch.

At this point we have some new files in our project. As of this post, my file structure looks like this. You’ll probably want to commit what you have right now to git.

If you run firebase serve in your terminal, it will host your website with the default files. And if you navigate to http://localhost:5000, you should see what the default files look like!

5. Configuring Firebase

The file firebase.json is where you configure your Firebase project. Right now mine looks like this:

{
  "hosting": {
    "public": "public",
    "ignore": [
      "firebase.json",
      "**/.*",
      "**/node_modules/**"
    ]
  }
}

We’re going to be editing the rewrites value of hosting in the json file. So add on to your firebase.json so that it looks like this:

{
  "hosting": {
    "public": "public",
    "ignore": [
      "firebase.json",
      "**/.*",
      "**/node_modules/**"
    ],
    "rewrites": [
      {
        "source": "/",
        "destination": "/index.html"
      }
    ]
  }
}

Rewrites contains an array of json objects under the hosting json object. Each object under rewrites needs to contain a source and a destination value.

Rewrites essentially tells firebase what to return at each url. So with the rewrite object above, if someone goes to our site and visits /, the root url, Firebase will return with index.html from our public folder. If a user tries to visit any other url but /, Firebase will return a 404.

If no rewrites exist for the requested path, Firebase will simply serve whatever matches the raw url. So if I added an about.html file to my public folder, users will have to be directed to yourwebsite.com/about.html to view it. If you completely remove rewrites like it was in the inital firebase.json, then all Firebase will still server whatever matches the raw url. But if you added in a rewrite object like this:

{
  "source": "/about",
  "destination": "/about.html"
}

Then everytime someone navigates to yourwebsite.com/about, they will be served about.html. This is an easy way to keep urls clean. You can add as many rewrites as you want.

Another interesting rewrite is this one.

{
  "source": "**",
  "destination": "/index.html"
}

What this means is that instead of returning a 404 if the request doesn’t exist, then Firebase will just return index.html. There’s a lot more you can do with rewrites, you can read more about them here

Once you’ve added some content to your website, go ahead and commit it to your master branch and then pull that branch into your release branch so that we can view your website on the web. An alternative to deploying is the command firebase deploy, but this isn’t needed with automatic deployment.

5. Connecting Firebase to your domain

After updating your release branch, navigate to the Firebase Console online and to the “Hosting” tab.

Once there you should see a page that looks like this:

If you click one one of the domains in the table, you’ll be redirected to a live version of your website. But those domains are ugly, so let’s add our own by clicking on “Add Custom Domain”. For this I’m using the domain test.dalenw.dev, which will not be live after this post.

Anyways after you hit continue, Firebase will list an IP address and a record type of A. Copy this IP address and navigate to your DNS provider, probably Cloudflare if you set your domain up with them. Regardless, adding in a new DNS record will probably look like this:

Save that, and finish the setup on the Firebase Console. After waiting a couple minutes or so, if you navigate to your domain you’ll see your Firebase site!

Conclusion

So with this guide you setup a git repo, Firebase project, linked the two, and connected Firebase to your domain. Additionally, you can update your website automatically without having to run firebase deploy everytime you want to push your changes, and you have the start of a development pipeline by using multiple git branches. Enjoy!

If you want to, you can also enable Cloud Logging from the hosting tab or by going to your Project Settings > Integrations and clicking on Cloud Logging there. That way if a user ever reports an issue with your site, you can naviate there to view the logs and narrow down exactly what happened to cause the issue.

Updates

Update March 24, 2022

Updating the DNS on Cloudflare after connecting it with Firebase can be a little tricky and may not immediately work. Once you’re sure you’ve done it right you may need to wait a few minutes to a day for your domain to properly display your website.

First Post!

First post! Took me about half an hour to figure out how to use jekyll and import a theme. We’ll see where this little blog goes. I’ll try to bring over some of the tools I wrote on the old site onto this website as well, but that’s far down on my list of things to do.

I may be backlogging posts, but the date of this post is the date this website was created.