Sep20

Scylla Monitoring Stack 2.0

Subscribe to Our Blog

Scylla Release

The Scylla team is pleased to announce the release of Scylla Monitoring Stack 2.0

Scylla Monitoring is an open source stack for Monitoring Scylla Enterprise and Scylla Open Source, based on Prometheus and Grafana.  Scylla Monitoring 2.0 stack supports:

  • Scylla Open Source versions 2.1, 2.2, 2.3
  • Scylla Enterprise versions 2017.x and 2018.x
  • Scylla Manager 1.1

Scylla Monitoring 2.0 brings many improvements, both in dashboard usability and the underlying stack, in particular, moving to a new version of Prometheus. Please read the Monitoring upgrade guide carefully before upgrading.

Enterprise users are welcome to contact us for proactive help in the upgrade process.

Open Source users are welcome to use the User Slack or User Mailing list for any questions you may have.

New in Monitoring 2.0

  • Move to Prometheus version 2.3.2.
    Scylla Monitoring stack 1.0 was based on Prometheus 1.x. Moving to
    Prometheus 2.x brings many improvements, mostly in the storage format. Note that Prometheus 2.x  is not backward compatible with Prometheus 1.x, which can make the monitoring stack upgrade process more complex. More here.

  • Support for Multi-cluster and multi-DC dashboards
    The Prometheus target files contain mapping information to map nodes to their respective data centers (DCs) and clusters. You can then use Prometheus to filter charts on the dashboard for either the cluster or DC by choosing DC or Cluster from the drop-down multi-select buttons. This is very useful in cases where you are using one monitoring stack to monitor more than one cluster.

Scylla Monitor 2.0 - selecting DCs

Example from prometheus/scylla_servers.yml, monitoring two clusters (cluster1, cluster2), the first with two DCs (dc1, dc2)

- targets:

- 172.17.0.1:9180
- 172.17.0.2:9180

labels:

cluster: cluster1
dc: dc1

- targets:

- 172.17.1.1:9180
- 172.17.2.2:9180

labels:

cluster: cluster1
dc: dc2

- targets:

- 172.17.10.1:9180
- 172.17.10.2:9180

labels:

cluster: cluster2

The same applies to prometheus/node_exporter_servers.yml, with node_exporter port (9100)

Note that the Monitoring stack uses the data provided in the target file (or service discovery), and not the Scylla cluster topology as presented in “nodetool status”. We plan to fix this gap in a future release. #139

  • Identify nodes by their IP.
    Node IPs are replacing hostname as node identifiers in all dashboards. This unifies the identifiers of Scylla and node_exporter(OS level) metrics.
    #244

  • Support for Scylla Open Source 2.3.
    Use start-all.sh -v 2.3 to start with Scylla 2.3 Grafana dashboards. You can use multiple dashboards at the same time, for example, start-all.sh -v 2.2,2.3

    The following dashboards are available:

    • Scylla Overview Metrics 2.3
    • Scylla CPU Per Server Metrics 2.3
    • Scylla Per-Server Disk I/O 2.3
    • Scylla Per Server Metrics 2.3

  • Accept any instance name. Characters such as underscore and colon may be used for instance names for example, “127.0.0.1:9180” is a valid instance name. #351

 
Related Links     

 

Tzach LivyatanAbout Tzach Livyatan

Tzach Livyatan has a B.A. and MSc in Computer Science (Technion, Summa Cum Laude), and has had a 15 year career in development, system engineering and product management. In the past he worked in the Telecom domain, focusing on carrier grade systems, signalling, policy and charging applications.


Tags: grafana, Prometheus, scylla, Scylla Monitor, Scylla Monitoring Stack, ScyllaDB