Docker migrate 3 to 4

In Chirpstack 4 old NS and AS are replaced by one Chirpstack V4.

Idea is to add to docker-compose new Chirpstack V4 container with new postgres and new redis (all running under other names and on other ports). Additionally Debian based container is created only to run converter script. Converter script doesn't need any chirpstack services. It only needs access to old database and new database and to all config files.

See https://www.chirpstack.io/docs/v3-v4-migration.html

Steps:

• copy configuration/chirpstack from docker-compose.yml
• copy postgresql configuration to new directory configuration/postgresql_cs4
• add 3 new services do old docker-compose (based on docker-compose from CSv4). Old postgres and redis are left intact.
  • chirpstack - remember to pin it ot 4.8.1 which is supported by converter
  • postgres_cs4
  • redis_cs4
• Adapt host names and ports in environment variables.
• review and fill api secret in configuration/chirpstack/chirpstack.toml
• add v4_migrate=true to configuration/chirpstack/region_eu868.toml
• disable other regions in configuration/chirpstack/chirpstack.toml - See https://www.chirpstack.io/docs/v3-v4-migration.html if more regions needed.
• temporary replace env variables with host names in configuration/chirpstack/chirpstack.toml. Converter cannot parse env variables.
• build and place ./chirpstack-v3-to-v4 converter binary. See https://github.com/chirpstack/chirpstack-v3-to-v4
• backup old redis and postgres

In result:

docker-compose.yml

cat docker-compose.yml 
version: "3"

services:
  chirpstack:
    image: chirpstack/chirpstack:4.8.1
    command: -c /etc/chirpstack
    restart: unless-stopped
    volumes:
      - ./configuration/chirpstack:/etc/chirpstack
      - ./lorawan-devices:/opt/lorawan-devices
    depends_on:
      - postgres_cs4
      - mosquitto
      - redis_cs4
    environment:
      - MQTT_BROKER_HOST=mosquitto
      - REDIS_HOST=redis_cs4:6380
      - POSTGRESQL_HOST=postgres_cs4:5433
    ports:
      - 18080:8080

  debian:
    image: debian:bullseye-slim
    command: sleep 60000
    volumes:
      - ./configuration/chirpstack:/etc/chirpstack
      - ./chirpstack-v3-to-v4:/opt/chirpstack-v3-to-v4
      - ./configuration/chirpstack-network-server:/etc/chirpstack-network-server
      - ./configuration/chirpstack-application-server:/etc/chirpstack-application-server

  postgres_cs4:
    image: postgres:14-alpine
    restart: unless-stopped
    volumes:
      - ./configuration/postgresql_cs4/initdb:/docker-entrypoint-initdb.d
      - postgresqldata_cs4:/var/lib/postgresql/data
    environment:
      - POSTGRES_PASSWORD=root
      - PGPORT=5433

  redis_cs4:
    image: redis:7-alpine
    restart: unless-stopped
    command: redis-server --save 300 1 --save 60 100 --appendonly no --port 6380
    volumes:
      - redisdata_cs4:/data
 
 
#  chirpstack-network-server:
#    image: chirpstack/chirpstack-network-server:3
#    volumes:
#      - ./configuration/chirpstack-network-server:/etc/chirpstack-network-server
#    restart: always
#    depends_on:
#      - postgresql
#      - mosquitto
 
#  chirpstack-application-server:
#    image: chirpstack/chirpstack-application-server:3
#    ports:
#      - 8085:8080
#    volumes:
#      - ./configuration/chirpstack-application-server:/etc/chirpstack-application-server
#    restart: always
#    depends_on:
#      - chirpstack-network-server
 
 
#  chirpstack-gateway-bridge:
#    image: chirpstack/chirpstack-gateway-bridge:3
#    ports:
#      - 1700:1700/udp
#    volumes:
#      - ./configuration/chirpstack-gateway-bridge:/etc/chirpstack-gateway-bridge
#    restart: always
#    depends_on:
#      - mosquitto
 
#  chirpstack-geolocation-server:
#    image: chirpstack/chirpstack-geolocation-server:3
#    volumes:
#      - ./configuration/chirpstack-geolocation-server:/etc/chirpstack-geolocation-server
#    restart: always

  postgresql:
    image: postgres:9.6-alpine
    environment:
      - POSTGRES_PASSWORD=root
    volumes:
      - ./configuration/postgresql/initdb:/docker-entrypoint-initdb.d
      - postgresqldata:/var/lib/postgresql/data
    restart: always

  redis:
    image: redis:5-alpine
    volumes:
      - redisdata:/data
    restart: always

  mosquitto:
    image: eclipse-mosquitto:2
    ports:
      - 1883:1883
    volumes:
      - ./configuration/eclipse-mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf
    restart: always

volumes:
  postgresqldata:
  postgresqldata_cs4:
  redisdata:
  redisdata_cs4:

Start all containers: docker-compose up -d.

Chirpstack V4 is needed to seed new database. Without it converter is not working.

Then login into Debian helper container docker-compose exec debian /bin/bash and:

cd /opt
./chirpstack-v3-to-v4 \
--as-config-file /etc/chirpstack-application-server/chirpstack-application-server.toml \
--ns-config-file /etc/chirpstack-network-server/chirpstack-network-server.toml \
--cs-config-file /etc/chirpstack/chirpstack.toml \
--drop-tenants-and-users

2024/09/09 16:55:48 Reading ChirpStack configuration file: /etc/chirpstack/chirpstack.toml
2024/09/09 16:55:48 Reading AS configuration file: /etc/chirpstack-application-server/chirpstack-application-server.toml
2024/09/09 16:55:48 Start migration
2024/09/09 16:55:48 Deleting users and tenants from target database
2024/09/09 16:55:48 Migrating users
2024/09/09 16:55:48 Migrating organizations
2024/09/09 16:55:48 Migrating organization users
2024/09/09 16:55:48 Migrating applications
2024/09/09 16:55:48 Migrating application integrations
2024/09/09 16:55:48 Reading NS configuration file: /etc/chirpstack-network-server/chirpstack-network-server.toml
2024/09/09 16:55:48 Migrating region: EU868
2024/09/09 16:55:48 Migrating gateways
2024/09/09 16:55:48 Migrating gateway metrics
2024/09/09 16:55:48 Migrating device-profiles
2024/09/09 16:55:48 Migrating devices
2024/09/09 16:55:48 Migrating device-keys
2024/09/09 16:55:48 Migrating device-queue
2024/09/09 16:55:48 Migrating device metrics
2024/09/09 16:55:48 Migrate device <> gateway
2024/09/09 16:55:48 Migrating multicast-groups
2024/09/09 16:55:49 Migrating multicast-group devices
2024/09/09 16:55:49 Migrating multicast-group queue
2024/09/09 16:55:49 Done :)