| Branch: | Tag: | Revision:

root / doc / devnotes.rst @ 9110fb4a

History | View | Annotate | Download (8.3 kB)

1 832aef24 Michael Hanselmann
Developer notes
2 832aef24 Michael Hanselmann
3 832aef24 Michael Hanselmann
4 727ee1ec Iustin Pop
.. highlight:: shell-example
5 727ee1ec Iustin Pop
6 b2fc7ea1 Michael Hanselmann
Build dependencies
7 b2fc7ea1 Michael Hanselmann
8 b2fc7ea1 Michael Hanselmann
9 fc6075dd Agata Murawska
Most dependencies from :doc:`install-quick`, including ``qemu-img``
10 fc6075dd Agata Murawska
(marked there as optional) plus (for Python):
11 cbf3d64b Michael Hanselmann
12 c27ba1cc Michael Hanselmann
- `GNU make <>`_
13 c27ba1cc Michael Hanselmann
- `GNU tar <>`_
14 c27ba1cc Michael Hanselmann
- `Gzip <>`_
15 18e2b6e4 Iustin Pop
- `pandoc <>`_
16 fc6075dd Agata Murawska
- `python-epydoc <>`_
17 c27ba1cc Michael Hanselmann
- `python-sphinx <>`_
18 fc6075dd Agata Murawska
  (tested with version 1.1.3)
19 7a694e30 Thomas Thrainer
- `python-mock <>`_
20 242e6bdd Thomas Thrainer
  (tested with version 1.0.1)
21 c27ba1cc Michael Hanselmann
- `graphviz <>`_
22 1de45c78 Guido Trotter
- the `en_US.UTF-8` locale must be enabled on the system
23 77a180f6 Iustin Pop
- `pylint <>`_ and its associated
24 77a180f6 Iustin Pop
25 08366664 Michael Hanselmann
- `pep8 <>`_
26 6e71fbf6 Bernardo Dal Seno
- `PyYAML <>`_
27 77a180f6 Iustin Pop
28 5f7d4181 Jose A. Lopes
For older developement (Ganeti < 2.4) ``docbook`` was used instead of
29 fc6075dd Agata Murawska
30 fc6075dd Agata Murawska
31 77a180f6 Iustin Pop
Note that for pylint, at the current moment the following versions
32 727ee1ec Iustin Pop
must be used::
33 77a180f6 Iustin Pop
34 77a180f6 Iustin Pop
    $ pylint --version
35 ab6536ba Michele Tartara
    pylint 0.26.0,
36 ab6536ba Michele Tartara
    astng 0.24.1, common 0.58.3
37 b2fc7ea1 Michael Hanselmann
38 fc6075dd Agata Murawska
The same with pep8, other versions may give you errors::
39 fc6075dd Agata Murawska
40 fc6075dd Agata Murawska
     $ pep8 --version
41 ab6536ba Michele Tartara
42 fc6075dd Agata Murawska
43 ab6536ba Michele Tartara
Both these versions are the ones shipped with Ubuntu 13.04.
44 099e9213 Iustin Pop
45 27e336af Michael Hanselmann
To generate unittest coverage reports (``make coverage``), `coverage
46 27e336af Michael Hanselmann
<>`_ needs to be installed.
47 27e336af Michael Hanselmann
48 fc6075dd Agata Murawska
Installation of all dependencies listed here::
49 fc6075dd Agata Murawska
50 8d53117b Michele Tartara
     $ apt-get install python-setuptools automake git fakeroot
51 bd341bff Thomas Thrainer
     $ apt-get install pandoc python-epydoc graphviz python-sphinx
52 242e6bdd Thomas Thrainer
     $ apt-get install python-yaml
53 bd341bff Thomas Thrainer
     $ cd / && easy_install \
54 ab6536ba Michele Tartara
               logilab-astng==0.24.1 \
55 ab6536ba Michele Tartara
               logilab-common==0.58.3 \
56 ab6536ba Michele Tartara
               pylint==0.26.0 \
57 ab6536ba Michele Tartara
               pep8==1.3.3 \
58 242e6bdd Thomas Thrainer
               mock==1.0.1 \
59 fc6075dd Agata Murawska
60 fc6075dd Agata Murawska
61 77a180f6 Iustin Pop
For Haskell development, again all things from the quick install
62 77a180f6 Iustin Pop
document, plus:
63 77a180f6 Iustin Pop
64 77a180f6 Iustin Pop
- `haddock <>`_, documentation
65 77a180f6 Iustin Pop
  generator (equivalent to epydoc for Python)
66 77a180f6 Iustin Pop
- `HsColour <>`_, again
67 77a180f6 Iustin Pop
  used for documentation (it's source-code pretty-printing)
68 77a180f6 Iustin Pop
- `hlint <>`_, a source code
69 3603605a Iustin Pop
  linter (equivalent to pylint for Python), recommended version 1.8 or
70 ab6536ba Michele Tartara
  above (tested with 1.8.43)
71 77a180f6 Iustin Pop
- the `QuickCheck <>`_
72 8e4f6d56 Iustin Pop
  library, version 2.x
73 95f6c931 Iustin Pop
- the `HUnit <>`_ library (tested with
74 95f6c931 Iustin Pop
75 95f6c931 Iustin Pop
- the `test-framework
76 95f6c931 Iustin Pop
  <>`_ libraries,
77 95f6c931 Iustin Pop
  tested versions: ``test-framework``: 0.6, ``test-framework-hunit``:
78 fe7ad9c8 Iustin Pop
  0.2.7, ``test-framework-quickcheck2``:
79 fd0bc853 Iustin Pop
- ``hpc``, which comes with the compiler, so you should already have
80 fd0bc853 Iustin Pop
81 727ee1ec Iustin Pop
- `shelltestrunner <>`_, used for
82 95f6c931 Iustin Pop
  running shell-based unit-tests
83 4355b2de Iustin Pop
- `temporary <>`_ library,
84 4355b2de Iustin Pop
  tested with version
85 727ee1ec Iustin Pop
86 727ee1ec Iustin Pop
Under Debian Wheezy or later, these can be installed (on top of the
87 727ee1ec Iustin Pop
required ones from the quick install document) via::
88 fd0bc853 Iustin Pop
89 95f6c931 Iustin Pop
  $ apt-get install libghc-quickcheck2-dev libghc-hunit-dev \
90 95f6c931 Iustin Pop
        libghc-test-framework-dev \
91 95f6c931 Iustin Pop
        libghc-test-framework-quickcheck2-dev \
92 95f6c931 Iustin Pop
        libghc-test-framework-hunit-dev \
93 bd341bff Thomas Thrainer
        libghc-temporary-dev shelltestrunner \
94 95f6c931 Iustin Pop
        hscolour hlint
95 fd0bc853 Iustin Pop
96 727ee1ec Iustin Pop
Or alternatively via ``cabal``::
97 727ee1ec Iustin Pop
98 21a5e56c Iustin Pop
  $ cabal install QuickCheck HUnit \
99 21a5e56c Iustin Pop
          test-framework test-framework-quickcheck2 test-framework-hunit \
100 4355b2de Iustin Pop
          temporary hscolour hlint shelltestrunner
101 77a180f6 Iustin Pop
102 b2fc7ea1 Michael Hanselmann
103 832aef24 Michael Hanselmann
Configuring for development
104 832aef24 Michael Hanselmann
105 832aef24 Michael Hanselmann
106 d17e74b4 Iustin Pop
Run the following command (only use ``PYTHON=...`` if you need to use a
107 d17e74b4 Iustin Pop
different python version)::
108 d17e74b4 Iustin Pop
109 727ee1ec Iustin Pop
  $ ./ && \
110 727ee1ec Iustin Pop
    ./configure --prefix=/usr/local --sysconfdir=/etc --localstatedir=/var
111 558fd122 Michael Hanselmann
112 cfa2b7a0 Iustin Pop
Note that doing development on a machine which already has Ganeti
113 cfa2b7a0 Iustin Pop
installed is problematic, as ``PYTHONPATH`` behaviour can be confusing
114 cfa2b7a0 Iustin Pop
(see Issue 170 for a bit of history/details; in general it works if
115 cfa2b7a0 Iustin Pop
the installed and developed versions are very similar, and/or if
116 cfa2b7a0 Iustin Pop
PYTHONPATH is customised correctly). As such, in general it's
117 cfa2b7a0 Iustin Pop
recommended to use a "clean" machine for ganeti development.
118 cfa2b7a0 Iustin Pop
119 9110fb4a Santi Raffa
Style guide
120 9110fb4a Santi Raffa
121 9110fb4a Santi Raffa
122 9110fb4a Santi Raffa
Please adhere to the :doc:`dev-codestyle` while writing code for Ganeti.
123 9110fb4a Santi Raffa
124 ef958f2a Iustin Pop
Haskell development notes
125 ef958f2a Iustin Pop
126 ef958f2a Iustin Pop
127 ef958f2a Iustin Pop
There are a few things which can help writing or debugging the Haskell
128 ef958f2a Iustin Pop
129 ef958f2a Iustin Pop
130 ef958f2a Iustin Pop
You can run the Haskell linter :command:`hlint` via::
131 ef958f2a Iustin Pop
132 727ee1ec Iustin Pop
  $ make hlint
133 ef958f2a Iustin Pop
134 3603605a Iustin Pop
This is not enabled by default (as the htools component is
135 3603605a Iustin Pop
optional). The above command will generate both output on the terminal
136 3603605a Iustin Pop
and, if any warnings are found, also an HTML report at
137 ef958f2a Iustin Pop
138 ef958f2a Iustin Pop
139 ef958f2a Iustin Pop
When writing or debugging TemplateHaskell code, it's useful to see
140 ef958f2a Iustin Pop
what the splices are converted to. This can be done via::
141 ef958f2a Iustin Pop
142 727ee1ec Iustin Pop
  $ make HEXTRA="-ddump-splices"
143 ef958f2a Iustin Pop
144 c96887cf Iustin Pop
Or, more interactively::
145 c96887cf Iustin Pop
146 c96887cf Iustin Pop
  $ ghci
147 c96887cf Iustin Pop
  λ> :set -ddump-splices
148 3add7574 Iustin Pop
  λ> :l src/Ganeti/Objects.hs
149 c96887cf Iustin Pop
150 c96887cf Iustin Pop
And you will get the spliced code as the module is loaded.
151 c96887cf Iustin Pop
152 d6b5da24 Guido Trotter
To build profiling code you must install the ``ghc-prof`` (or
153 d6b5da24 Guido Trotter
``gch6-prof``) package, and all the relevant libraries with their
154 d6b5da24 Guido Trotter
``-prof`` counterparts. If installing libraries through cabal the config
155 d6b5da24 Guido Trotter
file should include ``library-profiling: True`` or the ``-p`` flag
156 d6b5da24 Guido Trotter
should be used. Any library already installed can be updated by passing
157 d6b5da24 Guido Trotter
``--reinstall`` as well.
158 d6b5da24 Guido Trotter
159 ef958f2a Iustin Pop
Due to the way TemplateHaskell works, it's not straightforward to
160 c7ec3025 Iustin Pop
build profiling code. The recommended way is to run ``make hs-prof``,
161 c7ec3025 Iustin Pop
or alternatively the manual sequence is::
162 ef958f2a Iustin Pop
163 727ee1ec Iustin Pop
  $ make clean
164 3add7574 Iustin Pop
  $ make src/htools HEXTRA="-osuf .o"
165 3add7574 Iustin Pop
  $ rm src/htools
166 3add7574 Iustin Pop
  $ make src/htools HEXTRA="-osuf .prof_o -prof -auto-all"
167 ef958f2a Iustin Pop
168 ef958f2a Iustin Pop
This will build the binary twice, per the TemplateHaskell
169 ef958f2a Iustin Pop
documentation, the second one with profiling enabled.
170 ef958f2a Iustin Pop
171 ded769c1 Iustin Pop
The binary files generated by compilation and the profiling/coverage
172 ded769c1 Iustin Pop
files can "break" tab-completion in the sources; they can be ignored,
173 ded769c1 Iustin Pop
for example, in bash via ``.bashrc``::
174 ded769c1 Iustin Pop
175 ded769c1 Iustin Pop
176 ded769c1 Iustin Pop
177 ded769c1 Iustin Pop
or in emacs via ``completion-ignored-extensions`` (run ``M-x
178 ded769c1 Iustin Pop
customize-var completion-ignored-extensions``).
179 94338f0a Michael Hanselmann
180 06c9a9d6 Iustin Pop
Running individual tests
181 06c9a9d6 Iustin Pop
182 06c9a9d6 Iustin Pop
183 06c9a9d6 Iustin Pop
When developing code, running the entire test suite can be
184 5f7d4181 Jose A. Lopes
slow. Running individual tests is possible. There are different
185 5f7d4181 Jose A. Lopes
Makefile targets for running individual Python and Haskell tests.
186 06c9a9d6 Iustin Pop
187 06c9a9d6 Iustin Pop
For Python tests::
188 06c9a9d6 Iustin Pop
189 06c9a9d6 Iustin Pop
  $ export PYTHONPATH=$PWD
190 90066780 Iustin Pop
  $ python ./test/py/ganeti.%mytest%
191 06c9a9d6 Iustin Pop
192 06c9a9d6 Iustin Pop
For Haskell tests::
193 06c9a9d6 Iustin Pop
194 5f7d4181 Jose A. Lopes
  $ make hs-test-%pattern%
195 06c9a9d6 Iustin Pop
196 06c9a9d6 Iustin Pop
Where ``pattern`` can be a simple test pattern (e.g. ``comma``,
197 06c9a9d6 Iustin Pop
matching any test whose name contains ``comma``), a test pattern
198 06c9a9d6 Iustin Pop
denoting a group (ending with a slash, e.g. ``Utils/``), or more
199 5f7d4181 Jose A. Lopes
complex glob pattern. For more details, search for glob patterns in
200 5f7d4181 Jose A. Lopes
the documentation of `test-framework
201 06c9a9d6 Iustin Pop
202 06c9a9d6 Iustin Pop
203 5f7d4181 Jose A. Lopes
For individual Haskell shelltests::
204 5f7d4181 Jose A. Lopes
205 5f7d4181 Jose A. Lopes
  $ make hs-shell-%name%
206 5f7d4181 Jose A. Lopes
207 5f7d4181 Jose A. Lopes
which runs the test ``test/hs/shelltests/htools-%name%.test``. For
208 5f7d4181 Jose A. Lopes
example, to run the test ``test/hs/shelltests/htools-balancing.test``,
209 5f7d4181 Jose A. Lopes
210 5f7d4181 Jose A. Lopes
211 5f7d4181 Jose A. Lopes
  $ make hs-shell-balancing
212 5f7d4181 Jose A. Lopes
213 5f7d4181 Jose A. Lopes
For combined Haskell shelltests::
214 5f7d4181 Jose A. Lopes
215 5f7d4181 Jose A. Lopes
  $ make hs-shell-{%name1%,%name2%,...}
216 5f7d4181 Jose A. Lopes
217 5f7d4181 Jose A. Lopes
for example::
218 5f7d4181 Jose A. Lopes
219 5f7d4181 Jose A. Lopes
  $ make hs-shell-{balancing,basic}
220 5f7d4181 Jose A. Lopes
221 35bdbec3 Michele Tartara
Checking for the correct style of the NEWS file is also possible, by running::
222 35bdbec3 Michele Tartara
223 35bdbec3 Michele Tartara
  $ make check-news
224 35bdbec3 Michele Tartara
225 94338f0a Michael Hanselmann
Packaging notes
226 94338f0a Michael Hanselmann
227 94338f0a Michael Hanselmann
228 77865fb4 Gintautas Miliauskas
Ganeti is mostly developed and tested on `Debian
229 94338f0a Michael Hanselmann
<>`_-based distributions, while still keeping
230 77865fb4 Gintautas Miliauskas
adaptability to other Linux distributions in mind.
231 94338f0a Michael Hanselmann
232 94338f0a Michael Hanselmann
The ``doc/examples/`` directory contains a number of potentially useful
233 94338f0a Michael Hanselmann
scripts and configuration files. Some of them might need adjustment
234 94338f0a Michael Hanselmann
before use.
235 94338f0a Michael Hanselmann
236 94338f0a Michael Hanselmann
237 94338f0a Michael Hanselmann
238 94338f0a Michael Hanselmann
239 94338f0a Michael Hanselmann
This script, in the source code as ``daemons/``, is used
240 94338f0a Michael Hanselmann
to start/stop Ganeti and do a few other things related to system
241 77a180f6 Iustin Pop
daemons. It is recommended to use ``daemon-util`` also from the system's
242 94338f0a Michael Hanselmann
init scripts. That way the code starting and stopping daemons is shared
243 94338f0a Michael Hanselmann
and future changes have to be made in only one place.
244 94338f0a Michael Hanselmann
245 94338f0a Michael Hanselmann
``daemon-util`` reads extra arguments from variables (``*_ARGS``) in
246 94338f0a Michael Hanselmann
``/etc/default/ganeti``. When modifying ``daemon-util``, keep in mind to
247 94338f0a Michael Hanselmann
not remove support for the ``EXTRA_*_ARGS`` variables for starting
248 94338f0a Michael Hanselmann
daemons. Some parts of Ganeti use them to pass additional arguments when
249 94338f0a Michael Hanselmann
starting a daemon.
250 94338f0a Michael Hanselmann
251 94338f0a Michael Hanselmann
The ``reload_ssh_keys`` function can be adjusted to use another command
252 94338f0a Michael Hanselmann
for reloading the OpenSSH daemon's host keys.
253 94338f0a Michael Hanselmann
254 558fd122 Michael Hanselmann
.. vim: set textwidth=72 :