Statistics
| Branch: | Tag: | Revision:

root / doc / devnotes.rst @ 56c934da

History | View | Annotate | Download (8.2 kB)

1 832aef24 Michael Hanselmann
Developer notes
2 832aef24 Michael Hanselmann
===============
3 832aef24 Michael Hanselmann
4 727ee1ec Iustin Pop
.. highlight:: shell-example
5 727ee1ec Iustin Pop
6 b2fc7ea1 Michael Hanselmann
Build dependencies
7 b2fc7ea1 Michael Hanselmann
------------------
8 b2fc7ea1 Michael Hanselmann
9 fc6075dd Agata Murawska
Most dependencies from :doc:`install-quick`, including ``qemu-img``
10 fc6075dd Agata Murawska
(marked there as optional) plus (for Python):
11 cbf3d64b Michael Hanselmann
12 c27ba1cc Michael Hanselmann
- `GNU make <http://www.gnu.org/software/make/>`_
13 c27ba1cc Michael Hanselmann
- `GNU tar <http://www.gnu.org/software/tar/>`_
14 c27ba1cc Michael Hanselmann
- `Gzip <http://www.gnu.org/software/gzip/>`_
15 18e2b6e4 Iustin Pop
- `pandoc <http://johnmacfarlane.net/pandoc/>`_
16 fc6075dd Agata Murawska
- `python-epydoc <http://epydoc.sourceforge.net/>`_
17 c27ba1cc Michael Hanselmann
- `python-sphinx <http://sphinx.pocoo.org/>`_
18 fc6075dd Agata Murawska
  (tested with version 1.1.3)
19 7a694e30 Thomas Thrainer
- `python-mock <http://www.voidspace.org.uk/python/mock/>`_
20 242e6bdd Thomas Thrainer
  (tested with version 1.0.1)
21 c27ba1cc Michael Hanselmann
- `graphviz <http://www.graphviz.org/>`_
22 1de45c78 Guido Trotter
- the `en_US.UTF-8` locale must be enabled on the system
23 77a180f6 Iustin Pop
- `pylint <http://www.logilab.org/857>`_ and its associated
24 77a180f6 Iustin Pop
  dependencies
25 08366664 Michael Hanselmann
- `pep8 <https://github.com/jcrocholl/pep8/>`_
26 6e71fbf6 Bernardo Dal Seno
- `PyYAML <http://pyyaml.org/>`_
27 77a180f6 Iustin Pop
28 5f7d4181 Jose A. Lopes
For older developement (Ganeti < 2.4) ``docbook`` was used instead of
29 fc6075dd Agata Murawska
``pandoc``.
30 fc6075dd Agata Murawska
31 77a180f6 Iustin Pop
Note that for pylint, at the current moment the following versions
32 727ee1ec Iustin Pop
must be used::
33 77a180f6 Iustin Pop
34 77a180f6 Iustin Pop
    $ pylint --version
35 ab6536ba Michele Tartara
    pylint 0.26.0,
36 ab6536ba Michele Tartara
    astng 0.24.1, common 0.58.3
37 b2fc7ea1 Michael Hanselmann
38 fc6075dd Agata Murawska
The same with pep8, other versions may give you errors::
39 fc6075dd Agata Murawska
40 fc6075dd Agata Murawska
     $ pep8 --version
41 ab6536ba Michele Tartara
     1.3.3
42 fc6075dd Agata Murawska
43 ab6536ba Michele Tartara
Both these versions are the ones shipped with Ubuntu 13.04.
44 099e9213 Iustin Pop
45 27e336af Michael Hanselmann
To generate unittest coverage reports (``make coverage``), `coverage
46 27e336af Michael Hanselmann
<http://pypi.python.org/pypi/coverage>`_ needs to be installed.
47 27e336af Michael Hanselmann
48 fc6075dd Agata Murawska
Installation of all dependencies listed here::
49 fc6075dd Agata Murawska
50 8d53117b Michele Tartara
     $ apt-get install python-setuptools automake git fakeroot
51 bd341bff Thomas Thrainer
     $ apt-get install pandoc python-epydoc graphviz python-sphinx
52 242e6bdd Thomas Thrainer
     $ apt-get install python-yaml
53 bd341bff Thomas Thrainer
     $ cd / && easy_install \
54 ab6536ba Michele Tartara
               logilab-astng==0.24.1 \
55 ab6536ba Michele Tartara
               logilab-common==0.58.3 \
56 ab6536ba Michele Tartara
               pylint==0.26.0 \
57 ab6536ba Michele Tartara
               pep8==1.3.3 \
58 242e6bdd Thomas Thrainer
               mock==1.0.1 \
59 fc6075dd Agata Murawska
               coverage
60 fc6075dd Agata Murawska
61 77a180f6 Iustin Pop
For Haskell development, again all things from the quick install
62 77a180f6 Iustin Pop
document, plus:
63 77a180f6 Iustin Pop
64 77a180f6 Iustin Pop
- `haddock <http://www.haskell.org/haddock/>`_, documentation
65 77a180f6 Iustin Pop
  generator (equivalent to epydoc for Python)
66 77a180f6 Iustin Pop
- `HsColour <http://hackage.haskell.org/package/hscolour>`_, again
67 77a180f6 Iustin Pop
  used for documentation (it's source-code pretty-printing)
68 77a180f6 Iustin Pop
- `hlint <http://community.haskell.org/~ndm/hlint/>`_, a source code
69 3603605a Iustin Pop
  linter (equivalent to pylint for Python), recommended version 1.8 or
70 ab6536ba Michele Tartara
  above (tested with 1.8.43)
71 77a180f6 Iustin Pop
- the `QuickCheck <http://hackage.haskell.org/package/QuickCheck>`_
72 8e4f6d56 Iustin Pop
  library, version 2.x
73 95f6c931 Iustin Pop
- the `HUnit <http://hunit.sourceforge.net/>`_ library (tested with
74 95f6c931 Iustin Pop
  1.2.x)
75 95f6c931 Iustin Pop
- the `test-framework
76 95f6c931 Iustin Pop
  <http://batterseapower.github.com/test-framework/>`_ libraries,
77 95f6c931 Iustin Pop
  tested versions: ``test-framework``: 0.6, ``test-framework-hunit``:
78 fe7ad9c8 Iustin Pop
  0.2.7, ``test-framework-quickcheck2``: 0.2.12.1
79 fd0bc853 Iustin Pop
- ``hpc``, which comes with the compiler, so you should already have
80 fd0bc853 Iustin Pop
  it
81 727ee1ec Iustin Pop
- `shelltestrunner <http://joyful.com/shelltestrunner>`_, used for
82 95f6c931 Iustin Pop
  running shell-based unit-tests
83 4355b2de Iustin Pop
- `temporary <https://github.com/batterseapower/temporary/>`_ library,
84 4355b2de Iustin Pop
  tested with version 1.1.2.3
85 727ee1ec Iustin Pop
86 727ee1ec Iustin Pop
Under Debian Wheezy or later, these can be installed (on top of the
87 727ee1ec Iustin Pop
required ones from the quick install document) via::
88 fd0bc853 Iustin Pop
89 95f6c931 Iustin Pop
  $ apt-get install libghc-quickcheck2-dev libghc-hunit-dev \
90 95f6c931 Iustin Pop
        libghc-test-framework-dev \
91 95f6c931 Iustin Pop
        libghc-test-framework-quickcheck2-dev \
92 95f6c931 Iustin Pop
        libghc-test-framework-hunit-dev \
93 bd341bff Thomas Thrainer
        libghc-temporary-dev shelltestrunner \
94 95f6c931 Iustin Pop
        hscolour hlint
95 fd0bc853 Iustin Pop
96 727ee1ec Iustin Pop
Or alternatively via ``cabal``::
97 727ee1ec Iustin Pop
98 21a5e56c Iustin Pop
  $ cabal install QuickCheck HUnit \
99 21a5e56c Iustin Pop
          test-framework test-framework-quickcheck2 test-framework-hunit \
100 4355b2de Iustin Pop
          temporary hscolour hlint shelltestrunner
101 77a180f6 Iustin Pop
102 b2fc7ea1 Michael Hanselmann
103 832aef24 Michael Hanselmann
Configuring for development
104 832aef24 Michael Hanselmann
---------------------------
105 832aef24 Michael Hanselmann
106 d17e74b4 Iustin Pop
Run the following command (only use ``PYTHON=...`` if you need to use a
107 d17e74b4 Iustin Pop
different python version)::
108 d17e74b4 Iustin Pop
109 727ee1ec Iustin Pop
  $ ./autogen.sh && \
110 727ee1ec Iustin Pop
    ./configure --prefix=/usr/local --sysconfdir=/etc --localstatedir=/var
111 558fd122 Michael Hanselmann
112 cfa2b7a0 Iustin Pop
Note that doing development on a machine which already has Ganeti
113 cfa2b7a0 Iustin Pop
installed is problematic, as ``PYTHONPATH`` behaviour can be confusing
114 cfa2b7a0 Iustin Pop
(see Issue 170 for a bit of history/details; in general it works if
115 cfa2b7a0 Iustin Pop
the installed and developed versions are very similar, and/or if
116 cfa2b7a0 Iustin Pop
PYTHONPATH is customised correctly). As such, in general it's
117 cfa2b7a0 Iustin Pop
recommended to use a "clean" machine for ganeti development.
118 cfa2b7a0 Iustin Pop
119 ef958f2a Iustin Pop
Haskell development notes
120 ef958f2a Iustin Pop
-------------------------
121 ef958f2a Iustin Pop
122 ef958f2a Iustin Pop
There are a few things which can help writing or debugging the Haskell
123 ef958f2a Iustin Pop
code.
124 ef958f2a Iustin Pop
125 ef958f2a Iustin Pop
You can run the Haskell linter :command:`hlint` via::
126 ef958f2a Iustin Pop
127 727ee1ec Iustin Pop
  $ make hlint
128 ef958f2a Iustin Pop
129 3603605a Iustin Pop
This is not enabled by default (as the htools component is
130 3603605a Iustin Pop
optional). The above command will generate both output on the terminal
131 3603605a Iustin Pop
and, if any warnings are found, also an HTML report at
132 ef958f2a Iustin Pop
``doc/hs-lint.html``.
133 ef958f2a Iustin Pop
134 ef958f2a Iustin Pop
When writing or debugging TemplateHaskell code, it's useful to see
135 ef958f2a Iustin Pop
what the splices are converted to. This can be done via::
136 ef958f2a Iustin Pop
137 727ee1ec Iustin Pop
  $ make HEXTRA="-ddump-splices"
138 ef958f2a Iustin Pop
139 c96887cf Iustin Pop
Or, more interactively::
140 c96887cf Iustin Pop
141 c96887cf Iustin Pop
  $ ghci
142 c96887cf Iustin Pop
  λ> :set -ddump-splices
143 3add7574 Iustin Pop
  λ> :l src/Ganeti/Objects.hs
144 c96887cf Iustin Pop
145 c96887cf Iustin Pop
And you will get the spliced code as the module is loaded.
146 c96887cf Iustin Pop
147 d6b5da24 Guido Trotter
To build profiling code you must install the ``ghc-prof`` (or
148 d6b5da24 Guido Trotter
``gch6-prof``) package, and all the relevant libraries with their
149 d6b5da24 Guido Trotter
``-prof`` counterparts. If installing libraries through cabal the config
150 d6b5da24 Guido Trotter
file should include ``library-profiling: True`` or the ``-p`` flag
151 d6b5da24 Guido Trotter
should be used. Any library already installed can be updated by passing
152 d6b5da24 Guido Trotter
``--reinstall`` as well.
153 d6b5da24 Guido Trotter
154 ef958f2a Iustin Pop
Due to the way TemplateHaskell works, it's not straightforward to
155 c7ec3025 Iustin Pop
build profiling code. The recommended way is to run ``make hs-prof``,
156 c7ec3025 Iustin Pop
or alternatively the manual sequence is::
157 ef958f2a Iustin Pop
158 727ee1ec Iustin Pop
  $ make clean
159 3add7574 Iustin Pop
  $ make src/htools HEXTRA="-osuf .o"
160 3add7574 Iustin Pop
  $ rm src/htools
161 3add7574 Iustin Pop
  $ make src/htools HEXTRA="-osuf .prof_o -prof -auto-all"
162 ef958f2a Iustin Pop
163 ef958f2a Iustin Pop
This will build the binary twice, per the TemplateHaskell
164 ef958f2a Iustin Pop
documentation, the second one with profiling enabled.
165 ef958f2a Iustin Pop
166 ded769c1 Iustin Pop
The binary files generated by compilation and the profiling/coverage
167 ded769c1 Iustin Pop
files can "break" tab-completion in the sources; they can be ignored,
168 ded769c1 Iustin Pop
for example, in bash via ``.bashrc``::
169 ded769c1 Iustin Pop
170 ded769c1 Iustin Pop
  FIGNORE='.o:.hi:.prof_o:.tix'
171 ded769c1 Iustin Pop
172 ded769c1 Iustin Pop
or in emacs via ``completion-ignored-extensions`` (run ``M-x
173 ded769c1 Iustin Pop
customize-var completion-ignored-extensions``).
174 94338f0a Michael Hanselmann
175 06c9a9d6 Iustin Pop
Running individual tests
176 06c9a9d6 Iustin Pop
~~~~~~~~~~~~~~~~~~~~~~~~
177 06c9a9d6 Iustin Pop
178 06c9a9d6 Iustin Pop
When developing code, running the entire test suite can be
179 5f7d4181 Jose A. Lopes
slow. Running individual tests is possible. There are different
180 5f7d4181 Jose A. Lopes
Makefile targets for running individual Python and Haskell tests.
181 06c9a9d6 Iustin Pop
182 06c9a9d6 Iustin Pop
For Python tests::
183 06c9a9d6 Iustin Pop
184 06c9a9d6 Iustin Pop
  $ export PYTHONPATH=$PWD
185 90066780 Iustin Pop
  $ python ./test/py/ganeti.%mytest%
186 06c9a9d6 Iustin Pop
187 06c9a9d6 Iustin Pop
For Haskell tests::
188 06c9a9d6 Iustin Pop
189 5f7d4181 Jose A. Lopes
  $ make hs-test-%pattern%
190 06c9a9d6 Iustin Pop
191 06c9a9d6 Iustin Pop
Where ``pattern`` can be a simple test pattern (e.g. ``comma``,
192 06c9a9d6 Iustin Pop
matching any test whose name contains ``comma``), a test pattern
193 06c9a9d6 Iustin Pop
denoting a group (ending with a slash, e.g. ``Utils/``), or more
194 5f7d4181 Jose A. Lopes
complex glob pattern. For more details, search for glob patterns in
195 5f7d4181 Jose A. Lopes
the documentation of `test-framework
196 06c9a9d6 Iustin Pop
<http://batterseapower.github.com/test-framework/>`_).
197 06c9a9d6 Iustin Pop
198 5f7d4181 Jose A. Lopes
For individual Haskell shelltests::
199 5f7d4181 Jose A. Lopes
200 5f7d4181 Jose A. Lopes
  $ make hs-shell-%name%
201 5f7d4181 Jose A. Lopes
202 5f7d4181 Jose A. Lopes
which runs the test ``test/hs/shelltests/htools-%name%.test``. For
203 5f7d4181 Jose A. Lopes
example, to run the test ``test/hs/shelltests/htools-balancing.test``,
204 5f7d4181 Jose A. Lopes
use::
205 5f7d4181 Jose A. Lopes
206 5f7d4181 Jose A. Lopes
  $ make hs-shell-balancing
207 5f7d4181 Jose A. Lopes
208 5f7d4181 Jose A. Lopes
For combined Haskell shelltests::
209 5f7d4181 Jose A. Lopes
210 5f7d4181 Jose A. Lopes
  $ make hs-shell-{%name1%,%name2%,...}
211 5f7d4181 Jose A. Lopes
212 5f7d4181 Jose A. Lopes
for example::
213 5f7d4181 Jose A. Lopes
214 5f7d4181 Jose A. Lopes
  $ make hs-shell-{balancing,basic}
215 5f7d4181 Jose A. Lopes
216 35bdbec3 Michele Tartara
Checking for the correct style of the NEWS file is also possible, by running::
217 35bdbec3 Michele Tartara
218 35bdbec3 Michele Tartara
  $ make check-news
219 35bdbec3 Michele Tartara
220 94338f0a Michael Hanselmann
Packaging notes
221 94338f0a Michael Hanselmann
===============
222 94338f0a Michael Hanselmann
223 77865fb4 Gintautas Miliauskas
Ganeti is mostly developed and tested on `Debian
224 94338f0a Michael Hanselmann
<http://www.debian.org/>`_-based distributions, while still keeping
225 77865fb4 Gintautas Miliauskas
adaptability to other Linux distributions in mind.
226 94338f0a Michael Hanselmann
227 94338f0a Michael Hanselmann
The ``doc/examples/`` directory contains a number of potentially useful
228 94338f0a Michael Hanselmann
scripts and configuration files. Some of them might need adjustment
229 94338f0a Michael Hanselmann
before use.
230 94338f0a Michael Hanselmann
231 94338f0a Michael Hanselmann
``daemon-util``
232 94338f0a Michael Hanselmann
---------------
233 94338f0a Michael Hanselmann
234 94338f0a Michael Hanselmann
This script, in the source code as ``daemons/daemon-util.in``, is used
235 94338f0a Michael Hanselmann
to start/stop Ganeti and do a few other things related to system
236 77a180f6 Iustin Pop
daemons. It is recommended to use ``daemon-util`` also from the system's
237 94338f0a Michael Hanselmann
init scripts. That way the code starting and stopping daemons is shared
238 94338f0a Michael Hanselmann
and future changes have to be made in only one place.
239 94338f0a Michael Hanselmann
240 94338f0a Michael Hanselmann
``daemon-util`` reads extra arguments from variables (``*_ARGS``) in
241 94338f0a Michael Hanselmann
``/etc/default/ganeti``. When modifying ``daemon-util``, keep in mind to
242 94338f0a Michael Hanselmann
not remove support for the ``EXTRA_*_ARGS`` variables for starting
243 94338f0a Michael Hanselmann
daemons. Some parts of Ganeti use them to pass additional arguments when
244 94338f0a Michael Hanselmann
starting a daemon.
245 94338f0a Michael Hanselmann
246 94338f0a Michael Hanselmann
The ``reload_ssh_keys`` function can be adjusted to use another command
247 94338f0a Michael Hanselmann
for reloading the OpenSSH daemon's host keys.
248 94338f0a Michael Hanselmann
249 558fd122 Michael Hanselmann
.. vim: set textwidth=72 :