Add a function to get the current time in microseconds
In some cases we need higher resolution that seconds; I've settled onmicroseconds as that is what 'threadDelay' wants, for exactly, so it'seasier if we keep the same units.
Signed-off-by: Iustin Pop <iustin@google.com>...
Increase Haskell log file resolution
Currently the Haskell logging goes with the default time formatting,which lacks sub-second resolution. This is not good, as it's toocoarse.
The patch adds picoseconds to the log; they will contain usually toomany zeroes, but the other option is to use a log format that (for...
First part of confd timer changes
This patch changes the resolution of the timers: the watcher timergoes from 60s to 17s, and the polling-mode timer goes from 2 secondsto 250ms. The code changes a bit more due to the changes in the unitsof the various constants....
Change the polling thread sleep behaviour
Currently, the polling thread sleeps at the end of its body; theoriginal reason was that when the thread initially starts, we want toload the config immediately (in the old-style config load model).
However, now we explicitly load the config, so we can make this thread...
Start confd in notify mode if we can
This patch changes the default model—where confd is always started inpolling mode—to a model where if possible we enable inotify modebefore starting any of the background threads.
There are some tricky details here: if we enable inotify, we should...
Explicitly load the config at confd startup
Currently, confd main function doesn't load the config, but simplyforks the individual timer threads, which will—as a side effect-alsoload the configuration as soon as they start.
However, this makes codes to the startup more complex, so let's...
Fix type of 'node_whitelist' request parameter
If opportunistic_locking is used, then 'node_whitelist' parameter passedto the allocator is set to the LU's owned node locks. However, LU owned_lockshas type of 'set' while IReqInstanceAlloc expects type of...
hv_xen: Add test for CPU pinning configuration
Add a unittest for a function formatting CPU pinning information forXen's configuration.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
kvm: deduplicate 'get output' code
We had the same code twice, and were about to add a third time. Betterto collapse it into just one function.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
kvm: extract a regexp matching out of a for loop
kvm: remove last version-based feature detection
This was left behind because it required a different kvm invocation.Now that we can add new ones cheaply (two constants) it's easy to getrid of it. Differently than in other cases we support old version which...
Make Xen config path a build-time option
Stop hardcoding the path in “hv_xen.py”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Makefile.am: Protect placeholders for sed
If the placeholder has exactly the same name as the variable used inautoconf it would be replaced as well. In the case of“ENABLE_SPLIT_QUERY” this led to the sed command “s#False#False#g”.Inserting additional quotes after the first @ sign prevents this....
ENABLE_SPLIT_QUERY
burnin: Don't keep hypervisor class around
Just determine whether it can migrate and keep that value instead of thefull hypervisor class.
Text wrap in Haskell code
This adds two functions in Utils.hs: 'wrap' and 'trim' along with somesimple tests. 'wrap' is used in Common.hs for wrapping commanddescriptions to a fixed line length, e.g. useful for the "--help" option.
Signed-off-by: Yiannis Tsiouris <gtsiour@softlab.ntua.gr>...
Run pre-migrate hooks on primary node too
Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>Reviewed-by: Guido Trotter <ultrotter@google.com>
Makefile: Allow tarball creation when man-in-doc is active
Documentation with included man pages should not be distributed as itcontains fixed paths. Since the documentation with man pages is actuallywritten to a separate directory, tarball creation is still possible....
Check if KVM machine version is supported
If machine version is passed as an hv param, a check is madein target node whether this version is included in the supportedones derived from kvm -M ? command.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>...
Revert "run-in-tempdir: Improve speed"
This reverts commit dd60179dfd8 as it broke coverage for Python.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Verify that templates are compatible with exclusive storage
cluster-verify reports instances with disk templates not compatible withexclusive storage but that are running on nodes with the exclusive storageflag set.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
cluster-verify checks uniformity of PV sizes
When exclusive_storage is set, cluster-verify complains if the size of PVsis not uniform across a node group.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
New RPC call for node verify in dns-only RPC client
This will be used in node-add.
add-node checks PVs
add-node now performs a few checks on LVM before adding a node to thecluster.
Moved checks within LUClusterVerifyGroup
Almost all instance-specific checks have been moved from the Exec method tothe _VerifyInstance method. This cleans up Exec, which was becoming too bigeven for pylint…
bdev.GetPVInfo() returns list of LVs
This will be used for checks related to exclusive storage.
cluster-verify checks that PVs are not shared
When exclusive_storage is set, cluster-verify complains if PVs are sharedamong unrelated LVs.
Small cleanup in constants.py
- compat.UniqueFrozenset used in one more place, for consistency.- The order of NV_XXX constants is fixed so that all values are now sortedby name.
cluster-verify checks uniformity of exclusive_storage flag
The value of the flag should be the same for nodes within the same group.
utils.LvmExclusiveCheckNodePvs() returns more info
utils.LvmExclusiveCheckNodePvs() now returns the size of the smallest andbiggest PVs of a node. They will be used for cross-node consistency checks.
run-in-tempdir: Improve speed
By creating symlinks instead of actually creating files we still get thebenefits of using a temporary directory, but the preparation is aboutthree times as fast. Test used:
$ time bash -c 'for ((i=0; i < 100; ++i)); do autotools/run-in-tempdir true; done'...
run-in-tempdir: Copy doc/* only if needed
To preserve disk space on copying all files to a temporary directory,the “doc” directory is only copied when building documentation. Acomment describing the need is also added to Makefile.am, as well as asuperfluous direct invocation of Bash removed....
Integrate DRBD with the report generation infrastructure
With this commit, the DRBD data collector does not only print the datait extracts, but includes them in the proper JSON structure common toall the data collectors, as prescribed by the design document....
Add reporting infrastructure for data collectors
This commit adds the part of the JSON report generation code that will becommon to all the data collectors, according to the format specified in thedesign document.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Move getCurrentTime from Confd.Utils to Utils
The getCurrentTime function is of wide interest, so its moved to a moreappropriate position.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
burnin: Add unittest for supported disk templates
When new a new disk template is added, burnin should be updated.
kvm: make kvm path configurable as an hv parameter
This fixes Issue 225.
make _{Generate,Execute}KVMRuntime accept kvm help
Currently kvm --help is called twice at instance startup, this makessure it's called only once.
kvm: make auxiliary functions accept a kvm path
This will be used to be able to customize the path in an hv parameter.
Also fix two comments that still mentioned version-based checking.
Add KVM machine version as hv param
This is needed in case an instance must be booted with an exact type ofmachine version (due to e.g. outdated drivers). If this hv paramexplicitly defined (during instance creation/modification) then the kvmprocess will be started having this version for the -M option. Otherwise...
Move burnin to ganeti.tools.burnin
Similar to other scripts, “tools/burnin” just becomes a wrapper script.
Remove special-case of "0" in niceSort
Since now niceSort is more consistently behaved, we don't have thespecial casing. This should have been removed in commit a7f0953a, butI didn't realise it.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
burnin: Enable DT_EXT/DT_RBD for single node clusters
As reported by Constantinos Venetsanopoulos, DT_RBD and DT_EXT work onsingle node clusters.
test/Types.hs: add an HLint ANN for ignoring camelCase
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Link man pages in documentation
This patch depends on “Option to include man pages in documentation”. Inthe documentation build including man pages, all “:manpage:`…`”references are converted to links. For man pages not provided by Ganeti,Sphinx' standard formatting is used....
Make niceSort a wrapper over niceSortKey id
Currently, niceSort behaves differently than niceSortKey in caseswhere the parsed groups are equal but the original not, as it usesboth members of the tuple as opposed to just the first one. BothniceSortKey and the Python code use only the first one, so in corner...
Add version check for Sphinx
Sphinx 1.0 and above can check version by setting “needs_sphinx” in theconfiguration.
Option to include man pages in documentation
Before this patch, HTML versions of man pages (man/*.rst) were alreadybuilt. However, since they are separate from the normal documentation,their content is not indexed for Sphinx' search functionality.Additionally it would simply be nice to have everything in one place....
design-autorepair.rst: separate list of jobs with plus sign
Commas are not valid characters in tags, hence they can't be use toseparate the different job IDs; plus signs (+) are available, and not toobad.
burnin: Disk template option cleanup
Burnin doesn't support all disk templates (specifically it doesn'tsupport “blockdev”). The code, however, used manually composed lists insome places, constants.DISK_TEMPLATES in others, and an internal list ofsupported templates in some....
burnin: Factorize disk template lists
Keep the lists at the top instead of having them embedded in the code.Use frozenset, too.
Replace nodegroups' PartialNic by PartialNicParams
This fixes a bug that corrups the config when one adds anetwork to a node group. I wil soon extend QA by some commandsthat would have detected this.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Loader.hs: ignore expired ArSuspended policies
At the moment, because 'mergeData' is pure, it may set instance auto-repairpolicies that are of the form `ArSuspended $ Until timestamp_in_the_past`.If later on the auto-repair tool notices this, it has lost access to what...
Loader.hs: rewrite extractExTags to use chompPrefix
design-autorepair.rst: clarify tag precedence and conflict
This commit clarifies one particular point of the auto-repair workflow:what to do when multiple, conflicting administrator-set tags exist in anobject; and how tags at different levels (cluster, node group and instance)...
Loader.hs: set instance auto-repair policy in mergeData
'getArPolicy' and 'setArPolicy' follow the precedence rules introduced inb1eb71c: within an object, the most restrictive tag wins; across object,the nearest tag wins.
Signed-off-by: Dato Simó <dato@google.com>...
Instance.hs: add an 'arPolicy' field for auto-repair policy
Utils.hs: add a clockTimeToString function
This function allows to easily convert a ClockTime object to a stringrepresentation of its timestamp (seconds-only).
Utils.hs: function to chomp prefix + separator from a string
Some auto-repair tags are composed of a prefix and then an optionalargument; the new "chompPrefix" function in Utils.hs allows to strip aprefix, allowing the last character (the separator) to be absent if there...
design-autorepair.rst: add a graph with state transitions
The graph added here follows the data types introduced in 7ec2f76, by which"Suspended" is not a state, but a possible value of the auto-repair policy.
Add initial constants and Haskell ADTs for auto repair
In this commit, the AutoRepairType and AutoRepairResult types are defined,with the possible values specified in doc/design-autorepair.rst.
HTools/Types.hs: more auto-repair types
AutoRepairPolicy, AutoRepairStatus, and other auxiliary types are added.These are used only internally by the auto-repair tool, and parsed from thevarious object tags as defined in the design doc.
design-autorepair.rst: use the same prefix everywhere
This commit changes the "ganeti:watcher:repair:pending" and"ganeti:watcher:repair:result" tag prefixes to use"ganeti:watcher:autorepair:{pending,result}", so that the same prefix isused in all cases....
NEWS: Fix referenced man page's section number
kvm: add two help dump files
One is the --help output for kvm 1.1.2 and the second one is the same as0.9.1 but with a fake option added between -drive and its boot=on|offoption: this tests that if boot=on|off appears in another option after-drive it is not accepted by our regexp....
Update copyright in documentation
Use range() instead of building the list manually. Keeps the codeshorter starting next year.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-and-lolled-by: Iustin Pop <iustin@google.com>
kvm: simplify _GetMachineVersion
Use a multiline regexp rather than a single line one, and then splitlines in the code.
Allow -soundhw option to kvm
For all of us that love to listen to music on our kvm virtual machines,the -soundhw option is paramount.
kvm: support usb devices
All in one go, comma separated usb devices of all types.
kvm: Add kvm_extra parameter
As requested in Issue 261 and many other times.
kvm: Add -vga option
As requested in Issue 69.
kvm: use help based feature detection
As discussed on the list some people backport features to different kvmversions, making the version based detection system unreliable. Wechange wherever we can to detection on the help output.
This also fixes Issue 250...
Add a non negative int hypervisor parameter check
...and use it for a few parameters for which it makes sense.
kvm: always pass -usb
This will be the default anyway and it was a mistake to pass it only ifthe mouse was specified.
kvm: abstract getting --help output
This will be used later, for now nothing changes.
kvm: allow setting smp cores, threads, sockets
This was requested in Issue 322.Note that maxcpus is not implemented yet, as it should be considered ina cpu-hotplug global context.
kvm: add serial console speed
The serial port speed changed between versions of qemu (what didn't?), sothe old hardcoded default doesn't do anymore. Upgrade it to a parameter.
Merge branch 'devel-2.6'
Merge branch 'stable-2.6' into devel-2.6
QA: Use instance name for rename test
Reported in issue 341. In some setups the instance live in a differentnetblock from the cluster. Therefore a the configuration-global “rename”name shouldn't be used for them. Instead another instance name is used....
Add locking to _NetworkQuery
Make use of _GetNames() to get the final network UUIDs.Get networks info after having aquired the locks. Refactorthe code to use new local variables:
- "network_uuids" returned from _GetNames() - "all_networks" that are the latest network info...
QA: Factorize retrieving non-existent groups from config
Also add a check to make sure there are enough groups defined.
Generate documentation from build directory
When man pages should be included they need to be copied from man/*.rst.This means documentation can no longer be built from the static reSTfiles alone (which are referenced using “abs_top_srcdir”). Similar to...
Add machine version in kvm runtime file
kvm -M ? returns the supported machines (e.g. pc-1.1).Add _GetDefaultMachineVersion() function to get the default value.
Upon kvm runtime file creation (this is in _GenerateKVMRuntime() invokedonly in StartInstance()) append this info in kvm_cmd. During...
Fix typo in the DRBD parser types file
A missing letter has been added to a docstring.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add script to check man page references
This script checks for some of the most obvious mistakes when formattingman page references (which should have the form “**ganeti**\(7)”). Whilethis works now, it is very hard to avoid ambiguities (e.g. references...
man: Man page reference cleanup
Sphinx is stricter than pandoc when parsing reST and does not allow anopening parenthesis to directly follow inline markup. [1] describes therules. A backslash, as described in [2], can be used to separate thetwo.
Additionally, there were different forms for referring to man pages....
test/Utils.hs: fix capitalization in docstring comment
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Add utility to format dictionary as key=value strings
This will be used in QA to format network interface parameters.
This is a cherry-pick of master commit eac9b7b8
More documentation and man page fixes
- Include numbers in man page references- Use “:manpage:” where appropriate- Don't escape underscore in verbatim text (e.g. “``always_failover``) as it would show up in the generated man page- Quote whole paths, not just build-time part...
Remove fixed FIXME
This was fixed in stable-2.6, commit 053c356
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
QA: Support master-netdev and default NIC parameters
Requested in issue 337.
The parameter “bridge” was not documented and is therefore silentlyreplaced with “master-netdev”. A note is added to “qa-sample.json”describing how comments work.
This is a cherry-pick of master commit...
QA: Do not pass "--bridge" to "gnt-cluster init"
Commit ec0652a (June 2009) removed the option.
This is a cherry-pick of master commit784537390cd5b1c4f5f8fb9401a62eb23acce39b
Makefile: Fix list of directories
Commit dbee5c9 had a wrong path.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Highlight external links in HTML documentation
Based on MediaWiki's monobook skin.
Unit tests for bdev and utils.lvm
Also amended a comment of a tested method.The tests are for methods introduced in commit 63c73073 (LVM disk creationuses dedicated PVs).
Update the monitoring agent design document
This commit updates the design document of the monitoring agent accordingto what has already been discussed in various meetings and email threads.
Support instance-minor pairing in the DRBD collector
This commits enables the DRBD data collector to use the Confd client togather information about the pairing between DRBD minors and instances.
For testing purposes, the DRBD data collector now requires either zero...