527 |
527 |
it. For example a VM migration to a different physical node is transparent
|
528 |
528 |
to Cyclades.
|
529 |
529 |
|
530 |
|
|
531 |
530 |
Working with Cyclades
|
532 |
531 |
---------------------
|
533 |
532 |
|
... | ... | |
999 |
998 |
Cyclades advanced operations
|
1000 |
999 |
----------------------------
|
1001 |
1000 |
|
|
1001 |
.. _admin-guide-stats:
|
|
1002 |
|
|
1003 |
VM stats collecting
|
|
1004 |
~~~~~~~~~~~~~~~~~~~
|
|
1005 |
|
|
1006 |
snf-cyclades-gtools comes with a collectd plugin to collect CPU and network
|
|
1007 |
stats for Ganeti VMs and an example collectd configuration. snf-stats-app is a
|
|
1008 |
Django (snf-webproject) app that serves the VM stats graphs by reading the VM
|
|
1009 |
stats (from RRD files) and serves graphs.
|
|
1010 |
|
|
1011 |
The snf-stats-app was originally written by `GRNET NOC <http://noc.grnet.gr>`_
|
|
1012 |
as a WSGI Python app and was ported to a Synnefo (snf-webproject) app.
|
|
1013 |
|
|
1014 |
snf-stats-app configuration
|
|
1015 |
```````````````````````````
|
|
1016 |
|
|
1017 |
The snf-stats-app node should have collectd installed. The collectd
|
|
1018 |
configuration should enable the network plugin, assuming the server role, and
|
|
1019 |
the RRD plugin / backend, to store the incoming stats. Your
|
|
1020 |
``/etc/collectd/collectd.conf`` should look like:
|
|
1021 |
|
|
1022 |
.. code-block:: console
|
|
1023 |
|
|
1024 |
FQDNLookup true
|
|
1025 |
LoadPlugin syslog
|
|
1026 |
<Plugin syslog>
|
|
1027 |
LogLevel info
|
|
1028 |
</Plugin>
|
|
1029 |
|
|
1030 |
LoadPlugin network
|
|
1031 |
LoadPlugin rrdtool
|
|
1032 |
<Plugin network>
|
|
1033 |
TimeToLive 128
|
|
1034 |
<Listen "okeanos.io" "25826">
|
|
1035 |
SecurityLevel "Sign"
|
|
1036 |
AuthFile "/etc/collectd/passwd"
|
|
1037 |
</Listen>
|
|
1038 |
|
|
1039 |
ReportStats false
|
|
1040 |
MaxPacketSize 65535
|
|
1041 |
</Plugin>
|
|
1042 |
|
|
1043 |
|
|
1044 |
<Plugin rrdtool>
|
|
1045 |
DataDir "/var/lib/collectd/rrd"
|
|
1046 |
CacheTimeout 120
|
|
1047 |
CacheFlush 900
|
|
1048 |
WritesPerSecond 30
|
|
1049 |
RandomTimeout 0
|
|
1050 |
</Plugin>
|
|
1051 |
|
|
1052 |
Include "/etc/collectd/filters.conf"
|
|
1053 |
Include "/etc/collectd/thresholds.conf"
|
|
1054 |
|
|
1055 |
|
|
1056 |
An example collectd config file is provided in
|
|
1057 |
``/usr/share/doc/snf-stats-app/examples/stats-colletcd.conf``.
|
|
1058 |
|
|
1059 |
The recommended deployment is to run snf-stats-app using gunicorn with an
|
|
1060 |
Apache2 or nginx reverse proxy (using the same configuration as the other
|
|
1061 |
Synnefo services / apps). An example gunicorn config file is provided in
|
|
1062 |
``/usr/share/doc/snf-stats-app/examples/stats.gunicorn``.
|
|
1063 |
|
|
1064 |
Make sure to edit the settings under
|
|
1065 |
``/etc/synnefo/20-snf-stats-app-settings.conf`` to match your deployment.
|
|
1066 |
More specifically, you should change the ``STATS_BASE_URL`` setting (refer
|
|
1067 |
to previous documentation on the BASE_URL settings used by the other Synnefo
|
|
1068 |
services / apps) and the ``RRD_PREFIX`` and ``GRAPH_PREFIX`` settings.
|
|
1069 |
|
|
1070 |
You should also set the ``STATS_SECRET_KEY`` to a random string and make sure
|
|
1071 |
it's the same at the ``CYCLADES_STATS_SECRET_KEY`` on the Cyclades host (see
|
|
1072 |
below).
|
|
1073 |
|
|
1074 |
``RRD_PREFIX`` is the directory where collectd stores the RRD files. The
|
|
1075 |
default setting matches the default RRD directory for the collectd RRDtool
|
|
1076 |
plugin. In a more complex setup, the collectd daemon could run on a separate
|
|
1077 |
host and export the RRD directory to the snf-stats-app node via e.g. NFS.
|
|
1078 |
|
|
1079 |
``GRAPH_PREFIX`` is the directory where collectd stores the resulting
|
|
1080 |
stats graphs. You should create it manually, in case it doesn't exist.
|
|
1081 |
|
|
1082 |
.. code-block::
|
|
1083 |
|
|
1084 |
# mkdir /var/cache/snf-stats-app/
|
|
1085 |
# chown www-data:wwwdata /var/cache/snf-stats-app/
|
|
1086 |
|
|
1087 |
The snf-stats-app will typically run as the ``www-data`` user. In that case,
|
|
1088 |
make sure that the ``www-data`` user should have read access to the
|
|
1089 |
``RRD_PREFIX`` directory and read / write access to the ``GRAPH_PREFIX``
|
|
1090 |
directory.
|
|
1091 |
|
|
1092 |
snf-stats-app, based on the ``STATS_BASE_URL`` setting will export the
|
|
1093 |
following URL 'endpoints`:
|
|
1094 |
* CPU stats bar: ``STATS_BASE_URL``/v1.0/cpu-bar/<encrypted VM hostname>
|
|
1095 |
* Network stats bar: ``STATS_BASE_URL``/v1.0/net-bar/<encrypted VM hostname>
|
|
1096 |
* CPU stats daily graph: ``STATS_BASE_URL``/v1.0/cpu-ts/<encrypted VM hostname>
|
|
1097 |
* Network stats daily graph: ``STATS_BASE_URL``/v1.0/net-ts/<encrypted VM hostname>
|
|
1098 |
* CPU stats weekly graph: ``STATS_BASE_URL``/v1.0/cpu-ts-w/<encrypted VM hostname>
|
|
1099 |
* Network stats weekly graph: ``STATS_BASE_URL``/v1.0/net-ts-w/<encrypted VM hostname>
|
|
1100 |
|
|
1101 |
You can verify that these endpoints are exported by issuing:
|
|
1102 |
|
|
1103 |
.. code-block::
|
|
1104 |
|
|
1105 |
# snf-manage show_urls
|
|
1106 |
|
|
1107 |
snf-cyclades-gtools configuration
|
|
1108 |
`````````````````````````````````
|
|
1109 |
|
|
1110 |
To enable VM stats collecting, you will need to:
|
|
1111 |
* Install collectd on the every Ganeti (VM-capable) node.
|
|
1112 |
* Enable the Ganeti stats plugin in your collectd configuration. This can be
|
|
1113 |
achived by either copying the example collectd conf file that comes with
|
|
1114 |
snf-cyclades-gtools
|
|
1115 |
(``/usr/share/doc/snf-cyclades-gtools/examples/ganeti-stats-collectd.conf``)
|
|
1116 |
or by adding the following line to your existing (or default) collectd
|
|
1117 |
conf file:
|
|
1118 |
|
|
1119 |
Include /etc/collectd/ganeti-stats.conf
|
|
1120 |
|
|
1121 |
In the latter case, make sure to configure collectd to send the collected
|
|
1122 |
stats to your collectd server (via the network plugin). For more details on
|
|
1123 |
how to do this, check the collectd example config file provided by the
|
|
1124 |
package and the collectd documentation.
|
|
1125 |
|
|
1126 |
snf-cyclades-app configuration
|
|
1127 |
``````````````````````````````
|
|
1128 |
|
|
1129 |
At this point, stats collecting should be enabled and working. You can check
|
|
1130 |
that everything is ok by checking the contents of ``/var/lib/collectd/rrd/``
|
|
1131 |
directory (it will gradually get populated with directories containing RRD
|
|
1132 |
files / stats for every Synnefo instances).
|
|
1133 |
|
|
1134 |
You should also check that gunicorn and Apache2 are configured correctly by
|
|
1135 |
accessing the graph URLs for a VM (whose stats have been populated in
|
|
1136 |
``/var/lib/collectd/rrd``).
|
|
1137 |
|
|
1138 |
Cyclades uses the ``CYCLADES_STATS_SECRET_KEY`` setting in
|
|
1139 |
``20-snf-cyclades-app`` to encrypt the instance hostname in the stats graph
|
|
1140 |
URL. This settings should be set to a random value and match the
|
|
1141 |
``STATS_SECRET_KEY`` on the Stats host.
|
|
1142 |
|
|
1143 |
Cyclades (snf-cyclades-app) fetches the stat graphs for VMs based on four
|
|
1144 |
settings in ``20-snf-cyclades-app-api.conf``. The settings are:
|
|
1145 |
|
|
1146 |
* CPU_BAR_GRAPH_URL = 'https://stats.host/stats/v1.0/cpu-bar/%s'
|
|
1147 |
* CPU_TIMESERIES_GRAPH_URL = 'https://stats.host/stats/v1.0/cpu-ts/%s'
|
|
1148 |
* NET_BAR_GRAPH_URL = 'https://stats.host/stats/v1.0/net-bar/%s'
|
|
1149 |
* NET_TIMESERIES_GRAPH_URL = 'https://stats.host/stats/v1.0/net-ts/%s'
|
|
1150 |
|
|
1151 |
Make sure that you change this settings to match your ``STATS_BASE_URL``
|
|
1152 |
(and generally the Apache2 / gunicorn deployment on your stats host).
|
|
1153 |
|
|
1154 |
Cyclades will pass these URLs to the Cyclades UI and the user's browser will
|
|
1155 |
fetch them when needed.
|
|
1156 |
|
1002 |
1157 |
Reconciliation mechanism
|
1003 |
1158 |
~~~~~~~~~~~~~~~~~~~~~~~~
|
1004 |
1159 |
|