Some epydoc fixes
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add a hint to masterd for inconsistent clusters
Rightname confd's HMAC key
Currently, the ganeti-confd's HMAC key is called “cluster HMAC key” orsimply “HMAC key” everywhere. With the implementation of inter-clusterinstance moves, another HMAC key will be introduced for signing criticaldata. They can not be the same, so this patch clarifies the purpose of the...
daemon-util: Generate daemon path in separate function
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
daemon-util: Use “return” instead of “exit” in all functions
This is important if they're called directly within daemon-util.
daemon-util: Add function to start and stop all daemons
This is, to some degree, duplicated code from the init script. However,the init script has to conform to standards of the underlying Linuxdistributions, while these functions will be called by Ganeti itself. By...
ganeti.initd: Move all daemon names from init script to daemon-util
The list of daemon names will be used in daemon-util, too.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
ganeti.initd: Move code checking daemon exit code to daemon-util
This is again for re-using code.
ganeti.initd: Move code checking config to daemon-util
This allows for more code re-use. daemon-util will also be used to startall daemons.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
daemon-util: Require dashes in commands
Even though the script uses underscores (_) internally, the externalcommands are supposed to be written using dashes (-).
http.auth: Fix bug with checking hashed passwords
When username and password were sent for a resource not requiringauthentication, it wouldn't be accepted if the user in question had ahashed password. The reason was that the function GetAuthRealm used to...
Switch from os.path.join to utils.PathJoin
This passes a full burnin with lots of instances, and should be safe aswe mostly to join a known root (various constants) to a run-timevariable.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
watcher: Acquire lock early and give more friendly message
By opening the lock file early, other programs can lock thestate file to prevent ganeti-watcher from restarting daemons.Using the pause feature is inherently prone to race conditions.
Before a traceback was logged when the lock file couldn't...
Merge remote branch 'origin/devel-2.0' into devel-2.1
Conflicts: NEWS: Trivial configure.ac: Trivial...
Move watcher's EnsureDaemon function to utils
This is going to be used from the nbma repository, to ensure that thenld daemon is running.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add watcher hooks
These hooks are run on all nodes, after the "base" daemons are started.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Abstract starting the node daemons
We're using a separate function for this, as we're going to add somefunctionality to this feature.
ganeti-watcher: remove unused Indent function
Catch disk activation errors in watcher
If activating disks fails for some reason, the watcher didn'tcatch the exception. With this patch it's caught and logged.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Disable warning for not calling ProcessEvent init
This class doesn't need its constructor to be called.
Move RunInSeparateProcess to ganeti.utils
This function could be useful in other places and thisway we can easily unittest it.
ganeti-noded: Fix bug when export didn't succeed for all disks
snap_disks can contain boolean values. They weren't handled correctly.The error message was “Error while executing backend function: Invalidobject passed to FromDict: expected dict, got <type 'bool'>”....
Merge branch 'stable-2.1' into devel-2.1
Implement debug level across OS-related RPC calls
This doesn't implement the full functionality, we need to add the debuglevel to the opcodes too, but at least won't require changing the RPCcalls during the 2.1 series.
Signed-off-by: Iustin Pop <iustin@google.com>...
ganeti-cleaner: does 'echo 0' instead of 'exit 0'
This is from Debian Bug#568105.
ganeti-cleaner in one case erroneously calls "echo 0" instead of "exit0". As ganeti-cleaner is called in the daily cronjob, this leads to adaily mail containing just "0" with an unconfigured default install....
Merge remote branch 'origin/stable-2.1' into devel-2.1
ganeti-watcher: ensure confd is running as well
Ganeti-confd should be running on all 2.1 nodes.
node daemon: allow working with broken queue dir
In case the queue dir cannot be create/initialized, currentlyganeti-noded exits. This means that a read-only filesystem or apermission error breaks all node daemon functionality, includingpowercycle. This is not good for the usual failure case for nodes....
daemon-utils: remove usage of here-docs
In some versions of bash, here-docs and here-strings use temporaryfiles, which means daemon-util needs a writable temporary filesystem.Since echo is a bash builtin anyway, it's simple to switch to it andremove this dependency....
daemon-util: Fix quoting issue
This patch fixes a quoting issue in daemon-util:
The reason was that the generated variables were not quoted properly and...
workerpool: Make worker ID alphanumeric
Having a proper name instead of just a number makes debuggingeasier.
Introduce a Luxi call for GetTags
This changes from submitting jobs to get the tags (in cli scripts) toqueries, which (since the tags query is a cheap one) should be muchfaster.
The tags queries are already done without locks (in the generic querypaths for instances/nodes/cluster), so this shouldn't break tags query...
Fix unused imports or add silences where needed
In some cases pylint doesn't parse the import correctly, so we addsilences; but there are also many cases of unused imports, which wesimply remove.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
confd: add a TODO and a pylint disable
Two variables are used in a strange way in ExecConfd. Until that isclarified, add a TODO and a silence for the unused variable warning.
Further pylint disables, mostly for Unused args
Many of our functions have to follow a given API, and thus we have tokeep a given signature, but pylint doesn't understand this. Therefore,we silence this warning.
The patch does a few other cleanups.
daemons: handle arguments correctly and uniformly
Of all daemons, only rapi did abort when given argument. None of ourdaemons use any arguments, but they accepted them blindly. This is avery bad experience for the user.
This patch adds checking and exiting in all daemons, in a uniform way....
Remove more unused variables
This removes unused variables in the rest of the code (outside lib/).
Add targeted pylint disables
This patch should have only:
- pylint disables- docstring changes- whitespace changes
Rename an ConfdInotifyEventHandler init argument
'file' is a builtin keyword/type. Like many others, it should not beused as a variable/argument name.
No code is actually passing in this argument so renaming it is simple.
Fix use of the logging functions
The logging functions expand the arguments themselves, thus it's saferto let them do it rather than manual string formatting.
Also re-wraps one comment.
Change pyinotify import for broader compatibility
On some distributions pyinotify is installed in a different way, and theactual module just contains an internal pyinotify entry, which is theactual library. On others the main pyinotify module contains the library...
Remove quotes from CommaJoin and convert to it
This patch removes the quotes from CommaJoin and converts most of thecallers (that I could find) to it. Since CommaJoin does str(i) for i inparam, we can remove these, thus simplifying slightly a few calls....
Use “daemon-util” to reload SSH keys
Processor: support a unique execution id
When the processor is executing a job, it can export the execution id toits callers. This is not supported for Queries, as they're not executedin a job.
config.Add{Node,Instance}: get the ec id
This is ok because adding a node or instance cannot happen in a query.
We get the ec id from the LU and pass it to _EnsureUUID, which willthen for now not use it.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Add new “daemon-util” script to start/stop Ganeti daemons
Until now, Ganeti started and stopped its own daemons using custom functions.To start, the daemon was just executed and then sent the appropriate signals tostop it again. Init scripts would have to pay attention to the PID file and...
Convert the rest of the OpPrereqError users
This finishes the conversion of OpPrereqError creation to two-argumentstyle. Any leftovers as one-argument are not breaking anything, justlosing information about the errors.
ganeti-rapi: Use new function to verify passwords
This enables the use of hashed passwords in rapi_users.
Adding '--no-ssh-init' option to 'gnt-cluster init'.
Allows the initialization of a cluster without the creation or distributionof SSH key pairs. Includes changes for LeaveCluster and RPC.
Signed-off-by: Ken Wehr <ksw@google.com>Signed-off-by: Guido Trotter <ultrotter@google.com>...
Add timeout options to other LUs
All the LUs that shut down the instance need to be able too pass thetimeout parameter as well.
Accept shutdown timeout from the user
Using the new --timeout option:
- gnt-instance shutdown is changed to accept a timeout- the opcode is changed to hold one- the LU is changed to optionally get one- the rpc is changed to carry one- the backend is changed to take it as a parameter rather than...
Remove ‘-u’ from masterd shebang
This is not needed anymore - the original change was more than a yearago when masterd was in its incipient phase.
ganeti-confd: cleanup imports
Many functionalities of confd have been moved to other classes/modules,and the main confd daemon doesn't reference these modules directlyanymore.
ganeti-confd: don't depend on the os log dir
ganeti-confd doesn't need to log anything related to os installations.
Move the luxi error handling into errors.py
Currently the luxi error handling is hardcoded as special encoding onthe masterd-side and special decoding on the client side. This patchmoves it to errors.py such that other parts of the code can reuse thesame encoding....
Make ganeti-watcher use the standard debug option
Remove RpcResult.RemoteFailMsg completely
ganeti-confd: remove partial imports
ConfdAsyncUDPServer: handle signals at read time
Currently if a signal is delivered during an attempted read, anexception is logged in the logfile. There is no need for this, so wehandle this case explicitely.
ConfdAsyncUDPServer: defer handling writes
Currently if we fail writing to the socket (perhaps because a signal wasdelivered) we lose the data we were sending. Although this is not toobad (it's udp, and data may get lost anyway) we try to avoid this by...
Abstract AsyncUDPSocket to daemon
This allows this extended asyncore+udp module to be used also in otherdaemons, and in the confd client library
ConfdAsyncUDPServer: fix a docstring
It refers to an older input variable
Add a magic fourcc code to confd packets
This will make it easier to change the protocol later on
ganeti-confd: explicitely log failed big sends
Make sure that if we try to send packages which are too big (whichshouldn't happen) this gets properly logged in the config file.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Luca Bigliardi <shammash@google.com>
Move fourcc packing/unpacking to main confd module
This way it can be used by the client as well
Keep lock status with every job
This can be useful for debugging locking problems.
Move OpCode processor callbacks into separate class
There are two major arguments for this:- There will be more callbacks (e.g. for lock debugging) and extending the parameter list is a lot of work.- In the jqueue module this allows us to keep per-job or per-opcode variables in...
Wrap lines over 80 characters
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Luca Bigliardi <shammash@google.com>
gnt-cluster watcher: Show more information
confd: avoid spamming the logfile
When confd is disabled we don't want to be noticed every timer interval.
Merge branch 'next' into branch-2.1
Move SimpleConfigReader creation to ConfdProcessor
This will be useful to make ConfdProcessor aware of a config failure,without quitting confd.
ConfdProcessor: add disabled state
This is a state the processor will get in, if it fails to load theconfig.
confd: start in polling mode
This allows us not to enable the inotify handler immediately, and thusto make it easier for us should the config file not exist at all.
Confd: don't fail if the config doesn't load
Rather than quitting we'll just continue to poll the config at a slowrate, hoping that sooner or later we'll get it back. This allows alsoworking on non-MC nodes, and smoothly transitioning from MC to non-MC,...
confd: s/confd_event_handler/inotify_handler/
In a case we don't encounter frequently (file modified but notoverwritten) the notify handler we use is called with a wrong name.
Add script to clean archived jobs after 21 days
Implement timers in confd
Timers are used both for checking for inotify failures, and for polling,should inotify notices become too frequent.
ConfdInotifyEventHandler.enable: use InotifyError
Rather than raising ConfdFatalError directlyConfdInotifyEventHandler.enable raises InotifyError should it not beable to configure inotify, allowing the caller to decide what to do.
ConfdInotifyEventHandler, move to a callback
ConfdInotifyEventHandler used to reload the config whenever anotification arrived. Moving to a callback system, so thatConfdConfigurationReloader can be responsible for that functionality.
Additionally the inotify class no longer reenables itself automatically,...
Move creation of inotify handler to a new class
This class will be responsible for managing inotify notifications,timers, and rate-limiting reloads. For now none of these features isimplemented. :)
ConfdInotifyEventHandler: add enable/disable
Make possible to enable and disable the inotify event handler. Theinotify handler will remain enabled, unless explicitely told to disableitself.
ganeti-watcher: Don't run if paused
Add file to pause watcher for a certain duration
This can be used during maintenance work.
ganeti-masterd: Master voting in separate process
One shouldn't fork a Python process after using threads. Mastervoting is done before forking (utils.Daemonize), but it also usesthreads. Hence it's now called from a separate process.
This patch also fixes the check function to actually exit if...
ganeti-masterd: Add helper to run function in separate process
This will be used to do the master voting.
Style fixes for ganeti-*
Add disk copy support at backend and the rpc level
This uses a simple 'dd if=… | ssh $target dd of=…' method, like theExportSnapshot (which uses the OS export; here we want full disk-levelcopy and not any FS-level changes).
Convert ganeti-confd to Mainloop
Now that mainloop is asyncore-enabled we can easily do that.
Convert ganeti-masterd to @utils.SignalHandled
Add RPC call for storage operations
Merge commit 'origin/next' into branch-2.1
Remove unused imports from confd files
confd.server and daemons/ganeti-confd import a few modules they don'tactually use. Clean them up.
Remove a few unused imports from noded/masterd
Signed-off-by: Guido Trotter <ultrotter@google.com>
Initial confd implementation
ganeti-confd is a simple asynchronous daemon, which listens on a UDPport, passes each packet to a processor, and sends back to the clientthe result.
It also listens on an inotify socket, in order to reload itsconfiguration when the ganeti config file changes....
Merge branch 'master' into next
Handle None result from BlockdevFind
Use objects for blockdev_getmirrorstatus RPC call result
This patch changes the return type for backend.BlockdevGetmirrorstatus froma list of tuples to a list of objects.BlockDevStatus instances.
Use object for blockdev_find RPC call result
This patch changes the return type for backend.BlockdevFind to an object(objects.BlockDevStatus). Before a tuple was used. Adding more values tothis tuple causes a lot of work. Converting the result to an object with...