Usage Back to Top

Installation

We won’t cover further details how to properly setup Prometheus itself, we will only cover some basic setup based on docker-compose. But if you want to run this exporter without docker-compose you should be able to adopt that to your needs.

First of all we need to prepare a configuration for Prometheus that includes the exporter based on a static configuration with the container name as a hostname:

global:
  scrape_interval: 1m
  scrape_timeout: 10s
  evaluation_interval: 1m

scrape_configs:
- job_name: scw
  static_configs:
  - targets:
    - scw_exporter:9503

After preparing the configuration we need to create the docker-compose.yml within the same folder, this docker-compose.yml starts a simple Prometheus instance together with the exporter. Don’t forget to update the envrionment variables with the required credentials.

version: '2'

volumes:
  prometheus:

services:
  prometheus:
    image: prom/prometheus:latest
    restart: always
    ports:
      - 9090:9090
    volumes:
      - prometheus:/prometheus
      - ./prometheus.yml:/etc/prometheus/prometheus.yml

  scw_exporter:
    image: promhippie/scw-exporter:latest
    restart: always
    environment:
      - SCW_EXPORTER_ACESS_KEY=your-access-key
      - SCW_EXPORTER_SECRET_KEY=your-secret-key
      - SCW_EXPORTER_LOG_PRETTY=true

Since our latest tag always refers to the master branch of the Git repository you should always use some fixed version. You can see all available tags at DockerHub or Quay, there you will see that we also provide a manifest, you can easily start the exporter on various architectures without any change to the image name. You should apply a change like this to the docker-compose.yml file:

  scw_exporter:
-   image: promhippie/scw-exporter:latest
+   image: promhippie/scw-exporter:1.0.0
    restart: always
    environment:
      - SCW_EXPORTER_ACESS_KEY=your-access-key
      - SCW_EXPORTER_SECRET_KEY=your-secret-key
      - SCW_EXPORTER_LOG_PRETTY=true

If you want to access the exporter directly you should bind it to a local port, otherwise only Prometheus will have access to the exporter. For debugging purpose or just to discover all available metrics directly you can apply this change to your docker-compose.yml, after that you can access it directly at http://localhost:9503/metrics:

  scw-exporter:
    image: promhippie/scw-exporter:latest
    restart: always
+   ports:
+     - 127.0.0.1:9503:9503
    environment:
      - SCW_EXPORTER_ACESS_KEY=your-access-key
      - SCW_EXPORTER_SECRET_KEY=your-secret-key
      - SCW_EXPORTER_LOG_PRETTY=true

If you want to secure the access to the exporter you can provide a web config. You just need to provide a path to the config file in order to enable the support for it, for details about the config format look at the documentation section:

  scw_exporter:
    image: promhippie/scw-exporter:latest
    restart: always
    environment:
+     - SCW_EXPORTER_WEB_CONFIG=path/to/web-config.json
      - SCW_EXPORTER_ACESS_KEY=your-access-key
      - SCW_EXPORTER_SECRET_KEY=your-secret-key
      - SCW_EXPORTER_LOG_PRETTY=true

If you want to provide the required secrets from a file it’s also possible. For this use case you can write the secret to a file on any path and reference it with the following format:

  scw_exporter:
    image: promhippie/scw-exporter:latest
    restart: always
    environment:
-     - SCW_EXPORTER_ACESS_KEY=your-access-key
-     - SCW_EXPORTER_SECRET_KEY=your-secret-key
+     - SCW_EXPORTER_ACESS_KEY=file://path/to/secret/file/with/access-key
+     - SCW_EXPORTER_SECRET_KEY=file://path/to/secret/file/with/secret-key
      - SCW_EXPORTER_LOG_PRETTY=true

Finally the exporter should be configured fine, let’s start this stack with docker-compose, you just need to execute docker-compose up within the directory where you have stored the prometheus.yml and docker-compose.yml.

That’s all, the exporter should be up and running. Have fun with it and hopefully you will gather interesting metrics and never run into issues. You can access the exporter at http://localhost:9503/metrics and Prometheus at http://localhost:9090.

Configuration

SCW_EXPORTER_LOG_LEVEL
Only log messages with given severity, defaults to info
SCW_EXPORTER_LOG_PRETTY
Enable pretty messages for logging, defaults to false
SCW_EXPORTER_WEB_ADDRESS
Address to bind the metrics server, defaults to 0.0.0.0:9503
SCW_EXPORTER_WEB_PATH
Path to bind the metrics server, defaults to /metrics
SCW_EXPORTER_WEB_PPROF
Enable pprof debugging for server, defaults to false
SCW_EXPORTER_WEB_TIMEOUT
Server metrics endpoint timeout, defaults to 10s
SCW_EXPORTER_WEB_CONFIG
Path to web-config file
SCW_EXPORTER_REQUEST_TIMEOUT
Request timeout as duration, defaults to 5s
SCW_EXPORTER_ACESS_KEY
Access key for the Scaleway API
SCW_EXPORTER_SECRET_KEY
Secret key for the Scaleway API
SCW_EXPORTER_ORG
Organization for the Scaleway API
SCW_EXPORTER_PROJECT
Project for the Scaleway API
SCW_EXPORTER_REGION
Region for the Scaleway API
SCW_EXPORTER_ZONE
Zone for the Scaleway API
SCW_EXPORTER_COLLECTOR_DASHBOARD
Enable collector for dashboard, defaults to true
SCW_EXPORTER_COLLECTOR_SECURITY_GROUPS
Enable collector for security groups, defaults to true
SCW_EXPORTER_COLLECTOR_SERVERS
Enable collector for servers, defaults to true
SCW_EXPORTER_COLLECTOR_BAREMETAL
Enable collector for servers, defaults to true
SCW_EXPORTER_COLLECTOR_SNAPSHOTS
Enable collector for snapshots, defaults to true
SCW_EXPORTER_COLLECTOR_VOLUMES
Enable collector for volumes, defaults to true

Web Configuration

If you want to secure the service by TLS or by some basic authentication you can provide a YAML configuration file whch follows the Prometheus toolkit format. You can see a full configration example within the toolkit documentation.

Metrics

You can a rough list of available metrics below, additionally to these metrics you will always get the standard metrics exported by the Golang client of Prometheus. If you want to know more about these standard metrics take a look at the process collector and the Go collector.

scw_dashboard_images_count{zone}
Count of used images
scw_dashboard_ips_count{zone}
Count of used IPs
scw_dashboard_placement_groups_count{zone}
Count of placement groups
scw_dashboard_private_nics_count{zone}
Count of private nics
scw_dashboard_running_servers{zone}
Count of running servers
scw_dashboard_security_groups_count{zone}
Count of security groups
scw_dashboard_server_types_count{zone, type}
Count of servers by type
scw_dashboard_servers_count{zone}
Count of owned servers
scw_dashboard_snapshots_count{zone}
Count of used snapshots
scw_dashboard_unused_ips_count{zone}
Count of unused IPs
scw_dashboard_volumes_bssd_count{zone}
Count of unused IPs
scw_dashboard_volumes_bssd_total_size{zone}
Count of unused IPs
scw_dashboard_volumes_count{zone}
Count of used volumes
scw_dashboard_volumes_lssd_count{zone}
Count of unused IPs
scw_dashboard_volumes_lssd_total_size{zone}
Count of unused IPs
scw_request_duration_seconds{collector}
Histogram of latencies for requests to the api per collector
scw_request_failures_total{collector}
Total number of failed requests to the api per collector
scw_security_group_created_timestamp{id, name, zone, org, project}
Timestamp when the security group have been created
scw_security_group_defined{id, name, zone, org, project}
Constant value of 1 that this security group is defined
scw_security_group_enable_default{id, name, zone, org, project}
1 if the security group is enabled by default, 0 otherwise
scw_security_group_inbound_default_policy{id, name, zone, org, project}
1 if the security group inbound default policy is accept, 0 otherwise
scw_security_group_modified_timestamp{id, name, zone, org, project}
Timestamp when the security group have been modified
scw_security_group_outbound_default_policy{id, name, zone, org, project}
1 if the security group outbound default policy is accept, 0 otherwise
scw_security_group_project_default{id, name, zone, org, project}
1 if the security group is an project default, 0 otherwise
scw_security_group_servers_count{id, name, zone, org, project}
Number of servers attached to the security group
scw_security_group_stateful{id, name, zone, org, project}
1 if the security group is stateful by default, 0 otherwise
scw_server_created_timestamp{id, name, zone, org, project, type, private_ip, public_ip, arch}
Timestamp when the server have been created
scw_server_modified_timestamp{id, name, zone, org, project, type, private_ip, public_ip, arch}
Timestamp when the server have been modified
scw_server_private_nic_count{id, name, zone, org, project, type, private_ip, public_ip, arch}
Number of private nics attached
scw_server_state{id, name, zone, org, project, type, private_ip, public_ip, arch}
If 1 the server is running, depending on the state otherwise
scw_server_volume_count{id, name, zone, org, project, type, private_ip, public_ip, arch}
Number of volumes attached
scw_snapshot_available{id, name, zone, org, project}
Constant value of 1 that this snapshot is available
scw_snapshot_created_timestamp{id, name, zone, org, project}
Timestamp when the snapshot have been created
scw_snapshot_modified_timestamp{id, name, zone, org, project}
Timestamp when the snapshot have been modified
scw_snapshot_size_bytes{id, name, zone, org, project}
Size of the snapshot in bytes
scw_snapshot_state{id, name, zone, org, project}
State of the snapshot
scw_snapshot_type{id, name, zone, org, project}
Type of the snapshot
scw_volume_available{id, name, zone, org, project}
Constant value of 1 that this volume is available
scw_volume_created_timestamp{id, name, zone, org, project}
Timestamp when the volume have been created
scw_volume_modified_timestamp{id, name, zone, org, project}
Timestamp when the volume have been modified
scw_volume_size_bytes{id, name, zone, org, project}
Size of the volume in bytes
scw_volume_state{id, name, zone, org, project}
State of the snapshot
scw_volume_type{id, name, zone, org, project}
Type of the snapshot

Kubernetes Back to Top

Kubernetes

Currently we are covering the most famous installation methods on Kubernetes, you can choose between Kustomize and Helm.

Kustomize

We won’t cover the installation of Kustomize within this guide, to get it installed and working please read the upstream documentation. After the installation of Kustomize you just need to prepare a kustomization.yml wherever you like similar to this:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: scw-exporter

resources:
  - github.com/promhippie/scw_exporter//deploy/kubernetes?ref=master

configMapGenerator:
  - name: scw-exporter
    behavior: merge
    literals: []

secretGenerator:
  - name: scw-exporter
    behavior: merge
    literals: []

After that you can simply execute kustomize build | kubectl apply -f - to get the manifest applied. Generally it’s best to use fixed versions of the container images, this can be done quite easy, you just need to append this block to your kustomization.yml to use this specific version:

images:
  - name: quay.io/promhippie/scw-exporter
    newTag: 1.1.0

After applying this manifest the exporter should be directly visible within your Prometheus instance if you are using the Prometheus Operator as these manifests are providing a ServiceMonitor.

Helm

We won’t cover the installation of Helm within this guide, to get it installed and working please read the upstream documentation. After the installation of Helm you just need to execute the following commands:

helm repo add promhippie https://promhippie.github.io/charts
helm show values promhippie/scw-exporter
helm install scw-exporter promhippie/scw-exporter

You can also watch that available values and generally the details of the chart provided by us within our chart repository or on Artifacthub.

After applying this manifest the exporter should be directly visible within your Prometheus instance depending on your installation if you enabled the annotations or the service monitor.

Building Back to Top

As this project is built with Go you need to install Go first. The installation of Go is out of the scope of this document, please follow the official documentation. After the installation of Go you need to get the sources:

git clone https://github.com/promhippie/scw_exporter.git
cd scw_exporter/

All required tool besides Go itself are bundled, all you need is part of the Makfile:

make generate build

Finally you should have the binary within the bin/ folder now, give it a try with ./bin/scw_exporter -h to see all available options.

License Back to Top

This project is licensed under the Apache 2.0 license. For the license of the used libraries you have to check the respective sources.