Docker Swarm and Portainer Edge for edge deployments

Informatec Digital » Resources » Docker Swarm and Portainer Edge for edge deployments

When you start playing with Docker Swarm, Portainer, and edge deployments, it's easy to get confused: agents, Edge Agents, mesh routing, overlay networks, latency between VPSs in different cities… it all sounds great, but when it comes to setting it up, you want something clear, stable, and that doesn't force you to open half your firewall.

In the following lines you will find a detailed explanation in plain Spanish of how all this fits together: what Docker Swarm brings to the table, how mesh routing works, what role the Portainer Agent and Edge Agent play (in standard and asynchronous modes), how to manage distributed environments (for example, a VPS in New York and another in Seattle), and how to leverage Docker at the edge, even with WebAssembly (Wasm) for more limited devices.

Basic concepts: Docker Swarm, edge computing, and Portainer

Before we start tinkering with commands, it's important to understand some fundamental concepts that are common to any Docker Swarm edge architecture : Swarm as an orchestrator, the role of edge computing, and why Portainer makes life so much easier.

Docker Swarm is Docker's native clustering mode that lets you group multiple Docker Engines (for example, multiple VMs or VPSs) and manage them as if they were a single virtual host. In practical terms, this means you can easily deploy replicated services, use overlay networks between nodes, load balance, and leverage high availability.

On the other hand, edge computing involves bringing data processing closer to where the data is generated (sensors, IoT devices, small factory machines, routers, gateways, etc.). This reduces latency, saves bandwidth to the cloud, and allows operation even with limited or intermittent connectivity. Docker performs well on moderately powerful devices, but struggles on very limited hardware—for example, when using an immutable Linux distribution .

This is where Portainer comes in , a lightweight platform that offers a web interface and API for managing Docker, Swarm, Kubernetes, or ACI environments. Portainer lets you easily manage containers, images, networks, volumes, and other resources, whether on a single host or across dozens of remote nodes.

To connect these remote environments to your Portainer Server, you have several options: classic Portainer Agent, direct connection to the Docker API/socket, and Portainer Edge Agent in standard or asynchronous mode. Choosing the right option makes all the difference between a clean setup and one riddled with security vulnerabilities and open ports.

Docker Swarm: ports, mesh routing, and service publishing

When you enable Swarm mode in Docker , all nodes in the cluster participate in what's known as ingress routing mesh . In simpler terms, any node in the cluster can receive traffic on a service's published ports, even if that specific node isn't running any tasks for that service.

For all of this to work, certain ports must be open between the Swarm nodes . Specifically, the Swarm network plane uses:

• Port 7946 TCP/UDP for container network discovery.
• Port 4789 UDP (configurable) for VXLAN-based ingress network traffic.

Furthermore, the ports you publish for your services must be accessible from the external resources that will consume them : external load balancers, other systems, end users, etc. This traffic will enter through any node, and the mesh routing will be responsible for redirecting it to an active service container.

To publish ports in Swarm, the option is used. –publish When creating or updating a service. The parameter target indicates the internal port of the container, while published Define the port exposed in the routing mesh. If you skip the published port, Docker assigns a random high port to each task, and then you have to inspect the tasks to find out which one was used.

The published port (<PUBLISHED-PORT>) is where the Swarm exposes the service to the outside world, while the container port (<CONTAINER-PORT>This is where the process listens within the container. If, for example, you advertise external port 8080 pointing to the internal port 80 of an Nginx server, requesting any node on port 8080 will cause Swarm to route the request to one of the active containers without you having to worry about where it's actually running.

Mesh routing listens on the node's IP address and advertised port. If the IP address is externally routable, the service will be accessible from outside the host; if it's an internal-only IP address, access will be limited to the host itself or its private network. You can use docker service inspect to see at any time the relationship between the target ports (TargetPort) and those published (PublishedPort).

Publish TCP and/or UDP ports and use host mode

By default, when you publish a port on Swarm, it is assumed TCP protocolIf you want to publish UDP ports, or combine TCP and UDP, you must use the long syntax of --publish and adjust the key protocol a tcp o udpIf you publish both and omit the protocol, it will be published as TCP.

You can also skip mesh routing for a specific service using the so-called host modeIn this mode, when you access a port on a node, you reach only to the service instance running on that node, if there is one. To achieve this, you use the long syntax of --publish and you establish mode=hostIf you don't specify anything or leave mode=ingressThe mesh will be used.

With host mode, there are a couple of things to keep in mind to avoid surprises: if you access a node where there are no service tasks, that port may be idle or occupied by another application ; and if you expect to run multiple replicas per node, you won't be able to use a static port without conflicts, unless you let Docker assign random ports or limit it to a single replica per node using global services or constraints.

In larger architectures, it's common to combine Swarm with an external load balancer like HAProxy or similar. You can do this by leveraging mesh routing or using DNSRR (without VIPs) and balancing directly to the IPs of the tasks, depending on what best suits your traffic and observability needs.

Remote management with Portainer: Agent vs Edge Agent

Portainer offers several ways to connect remote Docker or Swarm environments to your Portainer Server , and this is where many people get confused. The three main options are: using the Portainer Agent, connecting directly to the Docker API/socket, or deploying the Portainer Edge Agent (standard or async mode).

The classic Portainer Agent is designed for environments where the Portainer Server and the managed hosts are on the same network or, at least, can communicate directly. In this scenario, the server connects to the agent, so you need to open ports from the server to the nodes running the agent, which isn't always desirable when dealing with VPSs distributed across multiple data centers or home networks behind NAT.

The most secure and flexible alternative in distributed environments is the Portainer Edge Agent . Here, the flow is reversed: the Edge Agent on the remote host establishes the connection to the Portainer Server. Instead of maintaining an open incoming connection, the Edge Agent periodically polls the server to check for pending jobs and acts accordingly.

For the Edge Agent to function, the Portainer Server must expose at least the web interface (UI) port , typically 9443 or a NodePort such as 30779 in Kubernetes. Additionally, in standard mode, you also need the tunnel port (8000 or 30776 in Kubernetes with NodePort). This tunnel creates a secure TLS connection between the Edge Agent and the server, enabling real-time interaction with the remote environment.

The official Portainer installation guides already configure the server to listen on both ports, but you'll need to ensure your environment's firewall exposes those ports. Portainer also assumes you meet certain minimum system requirements; if you deviate significantly from these, you may need additional adjustments or some functionality may be limited.

Edge Agent in standard mode on a Docker Swarm cluster

If you want to manage a remote Docker Swarm environment with a standard Edge Agent , the workflow in Portainer is usually something like this: from the Portainer menu, open the Environments section (Environment-related → Environments) and click on “Add environment”. There you choose “Docker Swarm” and start the wizard (Start Wizard).

Within the wizard, select the Edge Agent Standard option and fill in the environment details: descriptive name, URL to which the agent will connect, group options or tags, etc. In the "More settings" section, you can adjust the poll frequency , which determines how often the Edge Agent checks for new tasks or changes. By default, it's usually every 5 seconds, but you can modify it if you want to reduce traffic or improve responsiveness.

Once you've filled in the basic fields, click "Create." If you're pre-labeling environments or preparing automation scripts, you can obtain the join token at this point to reuse it in your deployments. Portainer will then display the additional fields you need to generate the installation command.

The next step is to choose the remote host platform (Linux or Windows), copy the command generated by Portainer, and run it in the Edge environment. This command downloads the Edge Agent image and configures it with the correct token and URL. Once it starts, the new environment appears on the Portainer main screen as another node.

If you need to deploy another similar standard Edge environment, you can use “Add another environment” and repeat the process. When you're finished with all the Edge environments, simply click “Continue” to proceed with configuring other environments or “Close” to return to the main list.

Edge Agent in async mode and with low quality connections

The Portainer Edge Agent's async mode is designed for scenarios where connectivity is limited, expensive, or highly unstable (mobile links ( 5G M2M connectivity ), rural connections, devices that connect only intermittently, etc.). In this mode, a real-time interactive tunnel is not established; instead, the Edge Agent sends status snapshots to the Portainer Server at configurable intervals.

With async, you can't open interactive sessions with the remote environment, but you can view the recent cluster state and launch actions based on that snapshot. Data consumption is much lower, making it ideal for edge deployments where every megabyte counts.

In async mode, the Edge Agent only needs the server to have the UI port (9443 or the equivalent NodePort) open. Tunnel port 8000 is no longer required, further simplifying the exposure of public ports. Setting up the environment in Portainer is very similar to standard mode; you simply select "Edge Agent Async" in the Docker Swarm environment wizard.

To add an Edge Async, go to Environments → Add environment, choose “Docker Swarm,” launch the wizard, select “Edge Agent Async,” and fill in the basic information. In “More settings,” you have three important intervals: Ping, Snapshot, and Command , which control the heartbeat frequency, the environment snapshot update, and the check for new commands, respectively. By default, everything is set to one minute, but you can adjust this as needed.

Just like in standard mode, you can use groups and tags to categorize Edge environments, create the environment, obtain the join token, and, once the command for your platform (Linux or Windows) is generated, run it on the remote host. Each async agent will register, and Portainer will be able to show you its status based on the latest snapshot received.

Typical scenario: two VPSs in different regions and use of Portainer

A fairly common case is that of someone who has two VPSs in different locations (for example, one in New York and one in Seattle) and wants to manage them with Portainer, using Docker Swarm or not, and setting up things like Prometheus, cAdvisor, and Node Exporter with private internal traffic between nodes.

In such a scenario, you wonder whether using only Swarm is sufficient or if you also need a Portainer Agent or an Edge Agent on each VPS. Docker Swarm by itself provides orchestration and overlay networking between nodes, allowing, for example, a Prometheus instance on the New York VPS to collect metrics from containers in Seattle via an encrypted overlay network.

Portainer, in this context, isn't mandatory but is highly recommended for centralizing the management of services, stacks, volumes, and networks . If you set up a Portainer Server on one of the VPSs (for example, in New York), you can connect the Seattle VPS with an Edge Agent. This way, you don't need to open the Docker API or maintain externally accessible ports for classic agents.

Regarding latency between Swarm nodes in different cities, it's typically in the tens of milliseconds, which in most cases isn't a serious problem for services like Prometheus, Node Exporter, or cAdvisor, which perform regular scraping. However, this should be taken into account if you're going to run services that are extremely sensitive to latency or consistency between nodes.

In summary, the most common and practical setup for this type of configuration is to use Docker Swarm for the cluster and overlay networks between the VPSs, and to use Portainer Edge Agent (better than the standard Agent) to securely manage the remote VPSs without opening unnecessary ports. The classic Agent makes sense when everything is on the same network or has very direct connectivity; in geographically distributed scenarios, Edge Agent is the logical choice.

Edge Jobs in Portainer on Docker Swarm

Another powerful feature when working with Portainer and Swarm environments is Edge Jobs . These jobs allow you to define tasks that will run in your Portainer-managed Edge environments, which is very useful for maintenance, reconfiguration, or periodic deployments.

A recurring question is whether it's possible to force an Edge Job to run on a specific node in the Swarm cluster. In many cases, the job always seems to run on one of the nodes, and you don't have clear control over which one it runs on, which can be confusing if you expect balanced behavior or want to target a specific node.

The key to controlling this lies in understanding how Portainer distributes jobs and how Swarm manages job placement . Depending on whether the job is launched as a standalone container on a node managed by Edge Agent or as a service within the Swarm cluster, you'll have more or less control through constraints or Edge Job configuration.

Although the out-of-the-box experience might give the impression that the node is chosen randomly, you can experiment with node labels, groups, and assignment rules to better determine where each job runs. This is where it's worthwhile to carefully review the Portainer documentation and conduct controlled tests until you have a clear understanding of the scheduling pattern.

Portainer at home: Edge Agent for controlling devices like Raspberry Pi

When we talk about using Portainer to manage home devices or small hosts like Raspberry Pi , the approach changes slightly. Here, Docker hosts are very often behind NAT, on home networks, and without a direct public IP address, making it inconvenient and risky to expose the Docker API or the classic Agent to the internet.

In this context, many users opt for the Portainer Edge Agent in standard mode to control their Raspberry Pi or other devices. This way, it's not necessary to modify the Docker configuration to enable the TCP API or open exposed sockets. The Edge Agent, as a reminder, opens the connection to the server, traversing NAT without you having to deal with port forwarding. Furthermore, these devices are commonly used for embedded vision projects with OpenMV and other edge applications.

The classic Portainer Agent requires the Portainer Server to be able to connect directly to the agents , something that isn't always feasible on a home network with CG-NAT or ISP-provided routers. Furthermore, it involves opening ports from outside your home network, which is generally not the most advisable approach in terms of security if you don't have a robust protection strategy in place.

With Edge Agent, on the other hand, you only need the server to have the UI port (and, in standard mode, the tunnel port) accessible from the internet or the network where your devices are located. Once the Edge Agents are connected, in Portainer's main view ("Home") you see each device as an independent environment, and you can use groups and tags to organize dozens of Raspberry Pis or edge devices without going crazy.

Async mode versus standard mode in edge environments

An important decision when choosing Portainer Edge Agent is whether to work in standard mode or async mode . Standard mode offers an experience very similar to having the host "almost local": you can open a console session in containers, view live logs, deploy stacks interactively, and so on.

Async mode, on the other hand, sacrifices interactivity to minimize bandwidth consumption and better tolerate erratic connections . In environments where you only want to check the status occasionally, launch specific jobs, and conserve data, async is the smart choice. In both cases, Portainer still allows you to categorize and manage your environments by groups and tags.

In terms of open ports, standard mode requires both the UI port and the tunnel port , while async mode only requires the UI port. If you're on a tight security path or want to minimize the exposed surface, async mode significantly simplifies the setup, at the cost of losing the interactive TLS tunnel.

Docker and WebAssembly (Wasm) for edge workloads

When we move down to the realm of edge computing on very limited devices (small ARMs, gateways with little RAM, etc.), even traditional containers can become somewhat cumbersome. This is where WebAssembly (Wasm) comes in, which already integrates natively with Docker through runtimes like WasmEdge or Wasmtime.

The advantages of Wasm for edge computing are clear: very small binaries, near-instantaneous startup, and strong capability-based isolation. Furthermore, portability across architectures (x86, ARM, RISC-V, etc.) is very high because the Wasm binary runs on the runtime, not directly on the operating system.

To use Wasm with Docker at the edge, you need a version of Docker that Wasm runtime supportDocker Desktop version 4.15 or later, or Docker Engine with containerd 1.7 or higher, usually work. Enabling this in daemon.json the containerd image store and declare the corresponding Wasm runtimes (for example, io.containerd.wasmedge.v1 o io.containerd.wasmtime.v1), and then restart the Docker daemon.

A typical example is a small service of sensor data processing Written in Rust and compiled to Wasm (target) wasm32-wasiThis program can read JSON data line by line, filter readings that exceed certain temperature or humidity thresholds, and issue alerts also in JSON, all with ridiculously low resource consumption.

Once compiled to WebAssembly, you can use a Minimalist, scratch-based Dockerfilecopying only the binary .wasm and declaring it as an ENTRYPOINT. When creating the image with docker buildx for the platform wasi/wasm And when you load it locally, you'll see how the resulting size is just a few megabytes compared to Python or Node images that easily reach a gigabyte.

Deploying Wasm at the edge with Docker and Compose

In real-world edge architectures, there are usually several services per device: sensor processors in WASM, MQTT brokers for communication, time series databases, lightweight reverse proxies, etc. Docker Compose is a perfect fit for bundling all of this into a single file.

An example of an edge node might have a Wasm service to process readings (using the corresponding Wasm runtime), an Eclipse Mosquitto broker to receive sensor data, a TimescaleDB/PostgreSQL database to store time series, and a Caddy as a reverse proxy to expose APIs or dashboards. Each service would have its ports, volumes, and always-on restart policies.

To facilitate distribution, it is common to set up a local Docker registration on the Edge Gateway itself, typically in port 5000. You package your Wasm image, the labels towards localhost:5000/... and you push it to the registry. From other nearby edge nodes you can pull from that local registry, reducing latency and resource consumption towards remote registries.

On very limited systems, it's also advisable to set strict resource restrictions : maximum memory (for example, 32 MB) and CPU limits (0.5 cores, etc.), even knowing that Wasm itself consumes little resources. This prevents a bug or a load spike from freezing the device and forcing you to manually restart.

Networks, Swarm, and Disconnection Tolerance at the Edge

At the edge, networks are typically less reliable than in a data center : frequent outages, IP changes, mobile links, etc. For this reason, it's common to create dedicated bridge networks at each node with custom subnets, or even build multi-site swarms linked via VPNs or secure tunnels.

With Docker you can create isolated bridge networks with their own bridge name on the host (for example, edge-br0), and then associate your services with that network to achieve greater separation from the rest of the traffic. If you take it a step further and set up Docker Swarm on the edge gateway, you can use overlay networks to connect containers between multiple nodes using the cluster-store and cluster-advertise mechanism (usually on Consul or another backend).

For multi-host scenarios with Swarm, Docker Machine remains a useful tool, as demonstrated by some deployments with OpenNebula . In that case, you can use docker-machine with an OpenNebula driver to launch VMs with boot2docker, set up a Consul for discovery, create the Swarm master and nodes, and finally connect everything with overlay networks.

The idea is simple: first, you create a VM for Consul, start it with the corresponding container on port 8500, obtain its IP address, and then use that IP address in the `docker-machine create` commands with the `--swarm` and `--swarm-discovery` options . With `cluster-store` and `cluster-advertise` configured, you can then create an overlay network and launch services (for example, Nginx on one node and BusyBox on another) that can communicate with each other using service names.

In edge environments that experience frequent disconnections, it's advisable to enable local persistence using volumes or bind mounts to store data on disk when connectivity is lost ( data cyber-resilience ). This allows your containers to accumulate data (for example, sensor readings) and send it in bulk when the connection is restored, following a "store-and-forward" pattern.

At the monitoring level, Docker offers a simple command like docker stats to obtain basic CPU, memory, and network traffic metrics. For more serious deployments, it's common to use lightweight agents such as Node Export along with Prometheus, even defined in a separate “monitoring” docker-compose, which expose host metrics to the central observability ecosystem.

Updating these edge deployments is also easier with Docker: you pull the new image, stop the old container, delete it, and restart the service with the new version. And, since Wasm containers start in milliseconds, the downtime is minimal, which is invaluable when the edge is collecting real-time data.

With this overview, it becomes clearer how the pieces fit together: Docker Swarm provides orchestration and overlay networking, Portainer Edge Agent simplifies remote management without opening risky ports, and Wasm reduces the cost of running applications on lean hardware . With proper planning of ports, networks, and agent modes (standard or async), you can build robust, distributed edge architectures across multiple locations, ready to scale without the daily administration headaches.