root / doc / devnotes.rst @ 9110fb4a
History | View | Annotate | Download (8.3 kB)
1 | 832aef24 | Michael Hanselmann | Developer notes |
---|---|---|---|
2 | 832aef24 | Michael Hanselmann | =============== |
3 | 832aef24 | Michael Hanselmann | |
4 | 727ee1ec | Iustin Pop | .. highlight:: shell-example |
5 | 727ee1ec | Iustin Pop | |
6 | b2fc7ea1 | Michael Hanselmann | Build dependencies |
7 | b2fc7ea1 | Michael Hanselmann | ------------------ |
8 | b2fc7ea1 | Michael Hanselmann | |
9 | fc6075dd | Agata Murawska | Most dependencies from :doc:`install-quick`, including ``qemu-img`` |
10 | fc6075dd | Agata Murawska | (marked there as optional) plus (for Python): |
11 | cbf3d64b | Michael Hanselmann | |
12 | c27ba1cc | Michael Hanselmann | - `GNU make <http://www.gnu.org/software/make/>`_ |
13 | c27ba1cc | Michael Hanselmann | - `GNU tar <http://www.gnu.org/software/tar/>`_ |
14 | c27ba1cc | Michael Hanselmann | - `Gzip <http://www.gnu.org/software/gzip/>`_ |
15 | 18e2b6e4 | Iustin Pop | - `pandoc <http://johnmacfarlane.net/pandoc/>`_ |
16 | fc6075dd | Agata Murawska | - `python-epydoc <http://epydoc.sourceforge.net/>`_ |
17 | c27ba1cc | Michael Hanselmann | - `python-sphinx <http://sphinx.pocoo.org/>`_ |
18 | fc6075dd | Agata Murawska | (tested with version 1.1.3) |
19 | 7a694e30 | Thomas Thrainer | - `python-mock <http://www.voidspace.org.uk/python/mock/>`_ |
20 | 242e6bdd | Thomas Thrainer | (tested with version 1.0.1) |
21 | c27ba1cc | Michael Hanselmann | - `graphviz <http://www.graphviz.org/>`_ |
22 | 1de45c78 | Guido Trotter | - the `en_US.UTF-8` locale must be enabled on the system |
23 | 77a180f6 | Iustin Pop | - `pylint <http://www.logilab.org/857>`_ and its associated |
24 | 77a180f6 | Iustin Pop | dependencies |
25 | 08366664 | Michael Hanselmann | - `pep8 <https://github.com/jcrocholl/pep8/>`_ |
26 | 6e71fbf6 | Bernardo Dal Seno | - `PyYAML <http://pyyaml.org/>`_ |
27 | 77a180f6 | Iustin Pop | |
28 | 5f7d4181 | Jose A. Lopes | For older developement (Ganeti < 2.4) ``docbook`` was used instead of |
29 | fc6075dd | Agata Murawska | ``pandoc``. |
30 | fc6075dd | Agata Murawska | |
31 | 77a180f6 | Iustin Pop | Note that for pylint, at the current moment the following versions |
32 | 727ee1ec | Iustin Pop | must be used:: |
33 | 77a180f6 | Iustin Pop | |
34 | 77a180f6 | Iustin Pop | $ pylint --version |
35 | ab6536ba | Michele Tartara | pylint 0.26.0, |
36 | ab6536ba | Michele Tartara | astng 0.24.1, common 0.58.3 |
37 | b2fc7ea1 | Michael Hanselmann | |
38 | fc6075dd | Agata Murawska | The same with pep8, other versions may give you errors:: |
39 | fc6075dd | Agata Murawska | |
40 | fc6075dd | Agata Murawska | $ pep8 --version |
41 | ab6536ba | Michele Tartara | 1.3.3 |
42 | fc6075dd | Agata Murawska | |
43 | ab6536ba | Michele Tartara | Both these versions are the ones shipped with Ubuntu 13.04. |
44 | 099e9213 | Iustin Pop | |
45 | 27e336af | Michael Hanselmann | To generate unittest coverage reports (``make coverage``), `coverage |
46 | 27e336af | Michael Hanselmann | <http://pypi.python.org/pypi/coverage>`_ needs to be installed. |
47 | 27e336af | Michael Hanselmann | |
48 | fc6075dd | Agata Murawska | Installation of all dependencies listed here:: |
49 | fc6075dd | Agata Murawska | |
50 | 8d53117b | Michele Tartara | $ apt-get install python-setuptools automake git fakeroot |
51 | bd341bff | Thomas Thrainer | $ apt-get install pandoc python-epydoc graphviz python-sphinx |
52 | 242e6bdd | Thomas Thrainer | $ apt-get install python-yaml |
53 | bd341bff | Thomas Thrainer | $ cd / && easy_install \ |
54 | ab6536ba | Michele Tartara | logilab-astng==0.24.1 \ |
55 | ab6536ba | Michele Tartara | logilab-common==0.58.3 \ |
56 | ab6536ba | Michele Tartara | pylint==0.26.0 \ |
57 | ab6536ba | Michele Tartara | pep8==1.3.3 \ |
58 | 242e6bdd | Thomas Thrainer | mock==1.0.1 \ |
59 | fc6075dd | Agata Murawska | coverage |
60 | fc6075dd | Agata Murawska | |
61 | 77a180f6 | Iustin Pop | For Haskell development, again all things from the quick install |
62 | 77a180f6 | Iustin Pop | document, plus: |
63 | 77a180f6 | Iustin Pop | |
64 | 77a180f6 | Iustin Pop | - `haddock <http://www.haskell.org/haddock/>`_, documentation |
65 | 77a180f6 | Iustin Pop | generator (equivalent to epydoc for Python) |
66 | 77a180f6 | Iustin Pop | - `HsColour <http://hackage.haskell.org/package/hscolour>`_, again |
67 | 77a180f6 | Iustin Pop | used for documentation (it's source-code pretty-printing) |
68 | 77a180f6 | Iustin Pop | - `hlint <http://community.haskell.org/~ndm/hlint/>`_, a source code |
69 | 3603605a | Iustin Pop | linter (equivalent to pylint for Python), recommended version 1.8 or |
70 | ab6536ba | Michele Tartara | above (tested with 1.8.43) |
71 | 77a180f6 | Iustin Pop | - the `QuickCheck <http://hackage.haskell.org/package/QuickCheck>`_ |
72 | 8e4f6d56 | Iustin Pop | library, version 2.x |
73 | 95f6c931 | Iustin Pop | - the `HUnit <http://hunit.sourceforge.net/>`_ library (tested with |
74 | 95f6c931 | Iustin Pop | 1.2.x) |
75 | 95f6c931 | Iustin Pop | - the `test-framework |
76 | 95f6c931 | Iustin Pop | <http://batterseapower.github.com/test-framework/>`_ libraries, |
77 | 95f6c931 | Iustin Pop | tested versions: ``test-framework``: 0.6, ``test-framework-hunit``: |
78 | fe7ad9c8 | Iustin Pop | 0.2.7, ``test-framework-quickcheck2``: 0.2.12.1 |
79 | fd0bc853 | Iustin Pop | - ``hpc``, which comes with the compiler, so you should already have |
80 | fd0bc853 | Iustin Pop | it |
81 | 727ee1ec | Iustin Pop | - `shelltestrunner <http://joyful.com/shelltestrunner>`_, used for |
82 | 95f6c931 | Iustin Pop | running shell-based unit-tests |
83 | 4355b2de | Iustin Pop | - `temporary <https://github.com/batterseapower/temporary/>`_ library, |
84 | 4355b2de | Iustin Pop | tested with version 1.1.2.3 |
85 | 727ee1ec | Iustin Pop | |
86 | 727ee1ec | Iustin Pop | Under Debian Wheezy or later, these can be installed (on top of the |
87 | 727ee1ec | Iustin Pop | required ones from the quick install document) via:: |
88 | fd0bc853 | Iustin Pop | |
89 | 95f6c931 | Iustin Pop | $ apt-get install libghc-quickcheck2-dev libghc-hunit-dev \ |
90 | 95f6c931 | Iustin Pop | libghc-test-framework-dev \ |
91 | 95f6c931 | Iustin Pop | libghc-test-framework-quickcheck2-dev \ |
92 | 95f6c931 | Iustin Pop | libghc-test-framework-hunit-dev \ |
93 | bd341bff | Thomas Thrainer | libghc-temporary-dev shelltestrunner \ |
94 | 95f6c931 | Iustin Pop | hscolour hlint |
95 | fd0bc853 | Iustin Pop | |
96 | 727ee1ec | Iustin Pop | Or alternatively via ``cabal``:: |
97 | 727ee1ec | Iustin Pop | |
98 | 21a5e56c | Iustin Pop | $ cabal install QuickCheck HUnit \ |
99 | 21a5e56c | Iustin Pop | test-framework test-framework-quickcheck2 test-framework-hunit \ |
100 | 4355b2de | Iustin Pop | temporary hscolour hlint shelltestrunner |
101 | 77a180f6 | Iustin Pop | |
102 | b2fc7ea1 | Michael Hanselmann | |
103 | 832aef24 | Michael Hanselmann | Configuring for development |
104 | 832aef24 | Michael Hanselmann | --------------------------- |
105 | 832aef24 | Michael Hanselmann | |
106 | d17e74b4 | Iustin Pop | Run the following command (only use ``PYTHON=...`` if you need to use a |
107 | d17e74b4 | Iustin Pop | different python version):: |
108 | d17e74b4 | Iustin Pop | |
109 | 727ee1ec | Iustin Pop | $ ./autogen.sh && \ |
110 | 727ee1ec | Iustin Pop | ./configure --prefix=/usr/local --sysconfdir=/etc --localstatedir=/var |
111 | 558fd122 | Michael Hanselmann | |
112 | cfa2b7a0 | Iustin Pop | Note that doing development on a machine which already has Ganeti |
113 | cfa2b7a0 | Iustin Pop | installed is problematic, as ``PYTHONPATH`` behaviour can be confusing |
114 | cfa2b7a0 | Iustin Pop | (see Issue 170 for a bit of history/details; in general it works if |
115 | cfa2b7a0 | Iustin Pop | the installed and developed versions are very similar, and/or if |
116 | cfa2b7a0 | Iustin Pop | PYTHONPATH is customised correctly). As such, in general it's |
117 | cfa2b7a0 | Iustin Pop | recommended to use a "clean" machine for ganeti development. |
118 | cfa2b7a0 | Iustin Pop | |
119 | 9110fb4a | Santi Raffa | Style guide |
120 | 9110fb4a | Santi Raffa | ----------- |
121 | 9110fb4a | Santi Raffa | |
122 | 9110fb4a | Santi Raffa | Please adhere to the :doc:`dev-codestyle` while writing code for Ganeti. |
123 | 9110fb4a | Santi Raffa | |
124 | ef958f2a | Iustin Pop | Haskell development notes |
125 | ef958f2a | Iustin Pop | ------------------------- |
126 | ef958f2a | Iustin Pop | |
127 | ef958f2a | Iustin Pop | There are a few things which can help writing or debugging the Haskell |
128 | ef958f2a | Iustin Pop | code. |
129 | ef958f2a | Iustin Pop | |
130 | ef958f2a | Iustin Pop | You can run the Haskell linter :command:`hlint` via:: |
131 | ef958f2a | Iustin Pop | |
132 | 727ee1ec | Iustin Pop | $ make hlint |
133 | ef958f2a | Iustin Pop | |
134 | 3603605a | Iustin Pop | This is not enabled by default (as the htools component is |
135 | 3603605a | Iustin Pop | optional). The above command will generate both output on the terminal |
136 | 3603605a | Iustin Pop | and, if any warnings are found, also an HTML report at |
137 | ef958f2a | Iustin Pop | ``doc/hs-lint.html``. |
138 | ef958f2a | Iustin Pop | |
139 | ef958f2a | Iustin Pop | When writing or debugging TemplateHaskell code, it's useful to see |
140 | ef958f2a | Iustin Pop | what the splices are converted to. This can be done via:: |
141 | ef958f2a | Iustin Pop | |
142 | 727ee1ec | Iustin Pop | $ make HEXTRA="-ddump-splices" |
143 | ef958f2a | Iustin Pop | |
144 | c96887cf | Iustin Pop | Or, more interactively:: |
145 | c96887cf | Iustin Pop | |
146 | c96887cf | Iustin Pop | $ ghci |
147 | c96887cf | Iustin Pop | λ> :set -ddump-splices |
148 | 3add7574 | Iustin Pop | λ> :l src/Ganeti/Objects.hs |
149 | c96887cf | Iustin Pop | |
150 | c96887cf | Iustin Pop | And you will get the spliced code as the module is loaded. |
151 | c96887cf | Iustin Pop | |
152 | d6b5da24 | Guido Trotter | To build profiling code you must install the ``ghc-prof`` (or |
153 | d6b5da24 | Guido Trotter | ``gch6-prof``) package, and all the relevant libraries with their |
154 | d6b5da24 | Guido Trotter | ``-prof`` counterparts. If installing libraries through cabal the config |
155 | d6b5da24 | Guido Trotter | file should include ``library-profiling: True`` or the ``-p`` flag |
156 | d6b5da24 | Guido Trotter | should be used. Any library already installed can be updated by passing |
157 | d6b5da24 | Guido Trotter | ``--reinstall`` as well. |
158 | d6b5da24 | Guido Trotter | |
159 | ef958f2a | Iustin Pop | Due to the way TemplateHaskell works, it's not straightforward to |
160 | c7ec3025 | Iustin Pop | build profiling code. The recommended way is to run ``make hs-prof``, |
161 | c7ec3025 | Iustin Pop | or alternatively the manual sequence is:: |
162 | ef958f2a | Iustin Pop | |
163 | 727ee1ec | Iustin Pop | $ make clean |
164 | 3add7574 | Iustin Pop | $ make src/htools HEXTRA="-osuf .o" |
165 | 3add7574 | Iustin Pop | $ rm src/htools |
166 | 3add7574 | Iustin Pop | $ make src/htools HEXTRA="-osuf .prof_o -prof -auto-all" |
167 | ef958f2a | Iustin Pop | |
168 | ef958f2a | Iustin Pop | This will build the binary twice, per the TemplateHaskell |
169 | ef958f2a | Iustin Pop | documentation, the second one with profiling enabled. |
170 | ef958f2a | Iustin Pop | |
171 | ded769c1 | Iustin Pop | The binary files generated by compilation and the profiling/coverage |
172 | ded769c1 | Iustin Pop | files can "break" tab-completion in the sources; they can be ignored, |
173 | ded769c1 | Iustin Pop | for example, in bash via ``.bashrc``:: |
174 | ded769c1 | Iustin Pop | |
175 | ded769c1 | Iustin Pop | FIGNORE='.o:.hi:.prof_o:.tix' |
176 | ded769c1 | Iustin Pop | |
177 | ded769c1 | Iustin Pop | or in emacs via ``completion-ignored-extensions`` (run ``M-x |
178 | ded769c1 | Iustin Pop | customize-var completion-ignored-extensions``). |
179 | 94338f0a | Michael Hanselmann | |
180 | 06c9a9d6 | Iustin Pop | Running individual tests |
181 | 06c9a9d6 | Iustin Pop | ~~~~~~~~~~~~~~~~~~~~~~~~ |
182 | 06c9a9d6 | Iustin Pop | |
183 | 06c9a9d6 | Iustin Pop | When developing code, running the entire test suite can be |
184 | 5f7d4181 | Jose A. Lopes | slow. Running individual tests is possible. There are different |
185 | 5f7d4181 | Jose A. Lopes | Makefile targets for running individual Python and Haskell tests. |
186 | 06c9a9d6 | Iustin Pop | |
187 | 06c9a9d6 | Iustin Pop | For Python tests:: |
188 | 06c9a9d6 | Iustin Pop | |
189 | 06c9a9d6 | Iustin Pop | $ export PYTHONPATH=$PWD |
190 | 90066780 | Iustin Pop | $ python ./test/py/ganeti.%mytest% |
191 | 06c9a9d6 | Iustin Pop | |
192 | 06c9a9d6 | Iustin Pop | For Haskell tests:: |
193 | 06c9a9d6 | Iustin Pop | |
194 | 5f7d4181 | Jose A. Lopes | $ make hs-test-%pattern% |
195 | 06c9a9d6 | Iustin Pop | |
196 | 06c9a9d6 | Iustin Pop | Where ``pattern`` can be a simple test pattern (e.g. ``comma``, |
197 | 06c9a9d6 | Iustin Pop | matching any test whose name contains ``comma``), a test pattern |
198 | 06c9a9d6 | Iustin Pop | denoting a group (ending with a slash, e.g. ``Utils/``), or more |
199 | 5f7d4181 | Jose A. Lopes | complex glob pattern. For more details, search for glob patterns in |
200 | 5f7d4181 | Jose A. Lopes | the documentation of `test-framework |
201 | 06c9a9d6 | Iustin Pop | <http://batterseapower.github.com/test-framework/>`_). |
202 | 06c9a9d6 | Iustin Pop | |
203 | 5f7d4181 | Jose A. Lopes | For individual Haskell shelltests:: |
204 | 5f7d4181 | Jose A. Lopes | |
205 | 5f7d4181 | Jose A. Lopes | $ make hs-shell-%name% |
206 | 5f7d4181 | Jose A. Lopes | |
207 | 5f7d4181 | Jose A. Lopes | which runs the test ``test/hs/shelltests/htools-%name%.test``. For |
208 | 5f7d4181 | Jose A. Lopes | example, to run the test ``test/hs/shelltests/htools-balancing.test``, |
209 | 5f7d4181 | Jose A. Lopes | use:: |
210 | 5f7d4181 | Jose A. Lopes | |
211 | 5f7d4181 | Jose A. Lopes | $ make hs-shell-balancing |
212 | 5f7d4181 | Jose A. Lopes | |
213 | 5f7d4181 | Jose A. Lopes | For combined Haskell shelltests:: |
214 | 5f7d4181 | Jose A. Lopes | |
215 | 5f7d4181 | Jose A. Lopes | $ make hs-shell-{%name1%,%name2%,...} |
216 | 5f7d4181 | Jose A. Lopes | |
217 | 5f7d4181 | Jose A. Lopes | for example:: |
218 | 5f7d4181 | Jose A. Lopes | |
219 | 5f7d4181 | Jose A. Lopes | $ make hs-shell-{balancing,basic} |
220 | 5f7d4181 | Jose A. Lopes | |
221 | 35bdbec3 | Michele Tartara | Checking for the correct style of the NEWS file is also possible, by running:: |
222 | 35bdbec3 | Michele Tartara | |
223 | 35bdbec3 | Michele Tartara | $ make check-news |
224 | 35bdbec3 | Michele Tartara | |
225 | 94338f0a | Michael Hanselmann | Packaging notes |
226 | 94338f0a | Michael Hanselmann | =============== |
227 | 94338f0a | Michael Hanselmann | |
228 | 77865fb4 | Gintautas Miliauskas | Ganeti is mostly developed and tested on `Debian |
229 | 94338f0a | Michael Hanselmann | <http://www.debian.org/>`_-based distributions, while still keeping |
230 | 77865fb4 | Gintautas Miliauskas | adaptability to other Linux distributions in mind. |
231 | 94338f0a | Michael Hanselmann | |
232 | 94338f0a | Michael Hanselmann | The ``doc/examples/`` directory contains a number of potentially useful |
233 | 94338f0a | Michael Hanselmann | scripts and configuration files. Some of them might need adjustment |
234 | 94338f0a | Michael Hanselmann | before use. |
235 | 94338f0a | Michael Hanselmann | |
236 | 94338f0a | Michael Hanselmann | ``daemon-util`` |
237 | 94338f0a | Michael Hanselmann | --------------- |
238 | 94338f0a | Michael Hanselmann | |
239 | 94338f0a | Michael Hanselmann | This script, in the source code as ``daemons/daemon-util.in``, is used |
240 | 94338f0a | Michael Hanselmann | to start/stop Ganeti and do a few other things related to system |
241 | 77a180f6 | Iustin Pop | daemons. It is recommended to use ``daemon-util`` also from the system's |
242 | 94338f0a | Michael Hanselmann | init scripts. That way the code starting and stopping daemons is shared |
243 | 94338f0a | Michael Hanselmann | and future changes have to be made in only one place. |
244 | 94338f0a | Michael Hanselmann | |
245 | 94338f0a | Michael Hanselmann | ``daemon-util`` reads extra arguments from variables (``*_ARGS``) in |
246 | 94338f0a | Michael Hanselmann | ``/etc/default/ganeti``. When modifying ``daemon-util``, keep in mind to |
247 | 94338f0a | Michael Hanselmann | not remove support for the ``EXTRA_*_ARGS`` variables for starting |
248 | 94338f0a | Michael Hanselmann | daemons. Some parts of Ganeti use them to pass additional arguments when |
249 | 94338f0a | Michael Hanselmann | starting a daemon. |
250 | 94338f0a | Michael Hanselmann | |
251 | 94338f0a | Michael Hanselmann | The ``reload_ssh_keys`` function can be adjusted to use another command |
252 | 94338f0a | Michael Hanselmann | for reloading the OpenSSH daemon's host keys. |
253 | 94338f0a | Michael Hanselmann | |
254 | 558fd122 | Michael Hanselmann | .. vim: set textwidth=72 : |