Add a new opcode timestamp field
Since the current start_timestamp opcode attribute refers to the initalstart time, before locks are acquired, it's not useful to determine theactual execution order of two opcodes/jobs competing for the same lock.
This patch adds a new field, exec_timestamp, that is updated when the...
Fix IgnoreSignals on socket.error
Some confusion arose handling EINTR on this function: in python 2.6socket.error is an IOError, and thus: - It's an EnvironmentError - It has an .errno member
In 2.4 and 2.5 it's not, and so its errno variable must be extracted...
RAPI client should convert urllib2.URLError to GanetiApiError
Signed-off-by: Tom Limoncelli <tlim@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
KVM: Migration bandwidth and downtime control
Introduce 2 new hypervisor options, migration_bandwidth and migration_downtimeand implement KVM migration bandwidth and downtime control.
migration_bandwidth controls KVM's maximal bandwidth during migration, in...
Make utils.EnsureDirs() ignore umask
EnsureDirs() should create directories with the exact mode requestedin the arguments, but it currently applies the umask.This patch makes it independent from the umask.
Signed-off-by: Balazs Lecz <leczb@google.com>...
Fix two race conditions in reboot instance
If the instance crashes between backend.InstanceReboot checks the listof running instances and the execution of hv_xen.RebootInstance,ini_info will be None. And if the instance doesn't reboot fast enough,new_info will be None. Both cases lead to “TypeError: unsubscriptable...
Support for latin friendly output in node list
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Test for errors during inotify callback
- Create a new _MyErrorLoggingAsyncNotifier class which registers error counts, rather than logging them- Add an additional ERR notifier to test with- Check that no error was returned, for tests that weren't supposed to...
SingleFileEventHandler: Remove try/except blocks
Since now we use the SingleFileEventHandler together with an errorhandling asyncore dispatcher, we don't need the internal try/exceptanymore.
ErrorLoggingAsyncNotifier
This mixes AsyncNotifier with GanetiBaseAsyncoreDispatcher to provide anAsyncNotifier which will log errors, rather than bail out.
daemon.GanetiBaseAsyncoreDispatcher
Abstract a few common functionalities between all ganeti asyncoredispatchers: - Handle errors by logging them, and then continue - By default check sockets only for readability
Signed-off-by: Guido Trotter <ultrotter@google.com>...
TestSingleFileEventHandler: abstract notifier type
Rather than hardcode that we have two notifiers, and notifier 0 is theterminating one, we abstract this with class level constants. This makesit easier to add more, with different features.
The only real change is that now the callback class takes as input the...
Mainloop: handle SIGINT as well (and terminate)
This is needed if daemons are in the foreground, and get ctrl+c-ed bythe user. Also add unittests to make sure the correct signals terminatethe mainloop.
SingleFileEventHandler: update comments
The comments in the SingleFileEventHandler are still confd-specific.Update them to make them generic for any single-file monitoring.
Add asyncnotifier unittests
Make ConfdInotifyEventHandler a library function
Cut&Paste, plus the following changes: - The class is renamed to SingleFileEventHandler - The monitored filename must be passed in and doesn't default to the ganeti cluster config file - A small docstring is added to the class...
Comment on AsyncNotifier upstream's availability
Since we contributed AsyncNotifier back to the upstream pyinotifyproject, we'll be able, one day, to remove the ganeti version of thatcode. For now we still need it to support older distributions, buthaving a note about when we'll be able to remove it is nice....
Remove errors.ConfdFatalError
This exception is caught, but never thrown. It became useless when wemoved confd from on/off to enabled/disabled, but always running on allnodes. Removing its definition and the code catching it can do no harm.
TestMainloop: test scheduler priorities as well
By using enterabs we can schedule events at the same time, which willthen be sorted by priority.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix race condition in mainloop unittest
Currently, in testDeferredCancel if the self._CancelEvent are enteredmore than 0.3 seconds after the _SendSig have been entered, the testcould fail. This is unlikely but may happen. To avoid it we useenterabs, to use absolute times....
TestMainloop.testReRun
Currently we never do that, but it's nice to know we can restart themainloop, after termination.
Add AsyncUDPSocket tests
Add initial mainloop unittests
devel/review: make the range argument optional
Rather than calling devel/review with from..to target we can just usetarget, and assume target..HEAD as the range. This makes it a loteasier, as now you just have to say something like "devel/reviewdevel-2.1" if you're already on the branch you want to commit, and want...
Abstract the LUXI eom into a constant
Currently the EOM terminator is hardcoded on the server side, and iscustomizable in the Transport object (with the default being the same asthe value found in the server), but not in the luxi client.
With this patch we move the value to constants, and remove the "fake"...
KVM: vhost net acceleration support
This will only work on patched or newer (>= 2.6.34) kernels and with apatched version of qemu-kvm.
ganeti-cleaner: Write log file with removed files
Logfiles can be useful for debugging.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
RAPI client: Handle urllib2.HTTPError and raise GanetiApiError
This allows users of the RAPI client to catch GanetiApiError for all HTTPerrors.
RAPI: /2/{nodes,instances}/$name should return 404 for unknown items
Currently they return a 500 Server Error, not really usefulfor detecting nonexistent items.
ganeti-masterd: Improve error logging for client requests
Ganeti errors should also be logged with a backtrace.
Return disk_template from LUQueryInstanceData
Inter-cluster instance moves need the disk template. As they runLUQueryInstanceData to get an instance's details, the disk templatemust be returned.
RAPI client: Rename Get{Node,Instance}Info, add new GetInstanceInfo
GetInstanceInfo should return the resource /2/instances/$name/info,but so far it returns /2/instances/$name. The same applies toGetNodeInfo, which returns /2/nodes/$name. Both names are stripped...
RAPI client: Log request to be made
Add missing documentation for RAPI instance creation mode
Add checks for master IP in cluster verify
This also updates a comment in the unittest for utils.py. We unittestthe new function for two things: correct reporting on real case (forlocalhost), and correct reporting with a mocked-out TcpPing that returns...
Remove unused import from daemon.py
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Luca Bigliardi <shammash@google.com>
Add a forgotten comment about overriding a method
AsyncUDPSocket.handle_error
By overriding the default asyncore handle_error (which closes thesocket) with our own version, which logs what happened but tries toproceed, we can get rid of a couple of try/except blocks. The resultingchurn is deindentation of the internal code....
utils.IgnoreSignals
Remove duplicate code between a couple of asyncore related function byhaving a function in charge of handling EINTR errors. Unittests included.
ganeti-noded: add the --no-mlock option
While mlock on noded is definitely good in most situations, there aresome - namely my laptop - where it has no benefit, and uses preciousnon-swappable memory. To avoid this we make it optional, with a new--no-mlock option. Note that only the main node daemon and its http...
Describe more ganeti-noded options in the manpage
Fix some pylint warnings
Disable warnings for:- except Exception,- use of __errno_location,- redeclaration of handleError()
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix logging string format
Fix this pylint warning:[W6501, Mlockall] Specify string format arguments as logging function parameters
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix Filehandler / FileHandler typo
Fix typo spotted by pylint:E1101:2095:LogFileHandler.handleError: Module 'logging' has no 'Filehandler' member
QA: Test instance creation/deletion via RAPI client
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
RAPI client: Implement instance creation
Currently this only supports the new instance creation request dataformat version 1, but support for the old version can be easilyimplemented.
Most arguments are optional and documented in the RAPI documentation....
RAPI: Add new request data format for instance creation
As mentioned in commit d975f482d, the current way of creating aninstance via RAPI is not very flexible. With this patch, a newinstance creation request data format is introduced and documented.Support can be detected by checking the list of features returned...
Mention RAPI client in documentation
rapi.baserlib: Add function to check variable type
Also add a separate function to retrieve body parameters. Havingit separate (independent of a class instance) will make it easierto unittest users of this function.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Add new /2/features RAPI resource
The /2/features RAPI resource can be used to detect optionalfeatures implemented by the RAPI server. This will be usedto recognize servers implementing a new request format forinstance creation requests.
Use console_logging in node daemon
Node daemon logs will be printed on system console if writing on the log filefails.
Daemons conditionally setup console logging
Use LogFileHandler conditionally in SetupLogging
Add a parameter to SetupLogging to use LogFileHandler (default is off)
Introduce LogFileHandler class
Add a log handler class which logs to /dev/console in case of logging errors.
Add /dev/console constant
Lock PowercycleNode child in memory
Lock node daemon children in memory
Lock node daemon in memory
Introduce Mlockall()
Add Mlockall() utility to lock current process' virtual adress space into RAM.
Node daemon availability improvements proposal
Fix wrong variable name in commit d975f482
RAPI: Add initial support for instance creation request version
The way the resource /2/instances expects its request data (e.g.instance name, disks, NICs) to be formatted in a dict is notvery extensible. HV and BE parameters are interleaved with allother values. In commit 495cfdf0 a new parameter “mode” was added...
Convert some ReadFile calls to ReadOneLineFile
For passwords we require strict oneliners, we're a bit more lax with pidand uid files.
utils.ReadOneLineFile()
Read the first non-empty file line. When strict is set, abort if morethan one line is non-empty. Some unittests inspired by the reverted onesfrom commit b774bb106cc28d008e790ad2666eb64c76866fa0, and some new ones.
Remove oneline= parameter from utils.ReadFile
This partially reverts commit b774bb106cc28d008e790ad2666eb64c76866fa0.Unittests unrelated to that particular functionality but introduced inthat commit are left untouched. Since the temporary directory is now...
INSTALL: update hypervisor requirements
Xen: Just specify 3.0 or above as a requirement, as versions above 3.1have been tested and work fine with Ganeti. Also note that the Xendependency is optional, if one wants to work with KVM.
KVM: list as an alternative, specify that 72 (the one shipped with...
install.rst: update os version and add KVM notes
The 0.7 version was just an example, but since we're on 0.9 now,updating the number cannot be wrong. The new admonition helps KVM usersto configure debootstrap correctly so that instances can be shutdown...
RAPI client: Fix SSL error reporting for real
My previous patch, commit 857705e8, helped in one particularsituation where the exception didn't contain any arguments(pyOpenSSL reporting a WantReadError, which shouldn't occur with ablocking socket anyway). With this patch, more common and uncommon...
RAPI client: Improve SSL error reporting
RAPI client: Don't check node role in client
Only the server knows which node roles can be set via RAPI.Constants are provided for convenience.
RAPI client: Update ReplaceInstanceDisks
- The RAPI client shouldn't check the replacement mode as only the server knows which ones are valid (constants are still provided for convenience)- Disks shouldn't be a list of names, but of indexes
RAPI client: Fix behaviour of “allocatable” storage flag
When modifying a storage unit, the “allocatable” flag should defaultto “no modification”. This replicates the behaviour of the commandline interface.
RAPI client: Encode boolean and None query values
Boolean values must be passed as 0 or 1. None should be an emptyvalue ("").
RAPI client: No longer check storage type locally
Only the RAPI server (actually masterd) knows which storage typesare valid. The exception can no longer be raised as the type isonly checked in the job.
RAPI client: Add constant for RAPI version
This reverts a60e3cb0a partially by moving the RAPI version into a constant.
RAPI QA: Use RAPI client
Add RAPI client utility module with RAPI PollJob function
The RAPI client module shouldn't depend on any Ganeti module, yet it'suseful to have some Ganeti-specific code, like a PollJob function forRAPI.
RAPI client: Don't assume job IDs to be numeric
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: David Knowles <dknowles@google.com>
QA: Add function create temporary backup file
QA: Restore RAPI cert after “gnt-cluster renew-crypto”
The RAPI tests will depend on the certificate.
RAPI client: Include HTTP error code in GanetiApiError exception
Having the HTTP error code allows users of the class to act differently basedon the error code.
Support different HTTP error codes in RAPI client unittest mock
This allows testing error conditions (e.g. if a job wasn't found).
RAPI client: Fix error message for unsupported methods in unittest
RAPI client: Allow waiting for job changes
RAPI: Allow waiting for job changes
RAPI client: Rename “DeleteJob” to “CancelJob”
Jobs can't be deleted, but cancelled (even though the HTTP method“DELETE” is used).
RAPI client: Various code style changes
- Replace hardcoded values with constants- Code formatting- Exception messages without periods and fixed string formatting
RAPI client: Add missing unittest for RepairNodeStorageUnits
RAPI client: Always pass _SendRequest args as positional
RAPI client: Simplify URL construction
RAPI client: Instantiate JSON encoder only once
RAPI client: Always return job ID
Even removing tags returns a job ID.
RAPI client: Hardcode version in URLs
If the version changes, the API is likely to change as well. Nothaving to ask for the version first makes the code simpler.
Remove httplib2 dependency from ganeti.rapi.client
- It's possible to implement all functionality in ganeti.rapi.client using Python's standard modules httplib and urllib2- By doing so, proper SSL certificate verification is implemented- Adjust some of the code to Ganeti's code style (this is not yet...
utils.ReadFile: Add parameter to read only one line
Reading only one line is useful for reading passwords from files. Thisalso adds unittests for utils.ReadFile.
Fix tiny typo in cluster verify
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
cli: Make PollJob generic to support other protocols
By separating the LUXI-specific code and stdio-related codeinto separate classes, we can make cli.PollJob protocol-agnostic, allowing it to be used with RAPI.
This patch also adds unittests for cli.PollJob....
Fix a unittest name and docstring
The docstring has been copied by the name, but the name reports amisspell of the function it is actually testing.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
NEWS: fix empty lines
Make sure there are exactly two empty lines between different versions(rather than 1 or 3).
Force ssh to allocate a tty
This is required to avoid the"Pseudo-terminal will not be allocated because stdin is not a terminal." ssh error message in case a Ganeti script is run non-interactively.
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Iustin Pop <iustin@google.com>