Change temporary disk from readonly to read-write
Making the temporary disk readonly has absolutely no advantages and ithas the disadvantage that it prevents the OS install helper fromsaving temporary data to it, for example, when downloading the OSinstall package which is necessary to run the OS scripts....
Fix docstring
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Remove Ganeti Lock Manager and Monitor
...from the jobs code, as this functionality is nowcarried out by wconfd. Also remove dead code resultingfrom this removal.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Demote a node from MCs only after the config. is updated
Otherwise it was possible that a node was demoted (by calling noded),its configuration copy removed, but before the configuration change tookplace, it was distributed to the node again.
Signed-off-by: Petr Pudlak <pudlak@google.com>...
Move configuration flush to a synchronized block
This ensures that the configuration is fully distributed and doesn'tchange during the verification process.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Let ConfigWriter delegate DRBD secrets to WConfd
Also WConfd handles proper synchronization internally so there is noneed to explicitly lock the configuration in ConfigWriter.
Merge branch 'stable-2.11' into stable-2.12
Merge branch 'stable-2.10' into stable-2.11
Forward MAC reservations from ConfigWriter to WConfD
Also call it for dropping reservations when a job finishes.
And remove the field for temporary DRBDs, that has been already moved toWConfD.
Fix lint error in 'masterd'
... and by that fixing a TODO as well.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Jose Lopes <jabolopes@google.com>
Fix gnt-network client wrt instances report
Let the gnt-network client expect a list of instance names and notUUIDs as returned by QueryNetworks (by both old and new style querymechanism).
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Helga Velroyen <helgav@google.com>
Fix QueryNetworks wrt instances
QueryNetworks tries to find which instances are connected to whichnetworks. The query mechanism in Haskell was written back when NICsreferred to a network via its name and not its UUID. Fix luxi tocomply with the current implementation (network slot of NIC object...
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Convert all the classes to new-style classes
... to make lint shut up.
Initialize self.needed_locks
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add --no-locks option to gnt-debug delay
Add the possibility to don't acquire locks during `gnt-debug delay`.This allows to run many delay jobs in parallel instead of havingthem run sequentially.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>...
Add a command to gnt-debug to flush the configuration
This allows to enforce it during QA tests.
Flush the configuration before cluster verify
For this expose the flushing operation in Python ConfigWriter.
Support group evacuation by failover
Support evacuating a node group not using migration.This can be useful if the group evacuated to has differenthardware.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Add an option --force-failover
...to be added to gnt-group evacuate forcing evacuation bymeans for failovers instead of migrations.
Don't synchronize the configuration on DropECReservations
The call doesn't modify the configuration state, only the reservations.And after moving reservations to WConfd, the reservations state will belocked just internally by WConfd.
Since this method is called at the end of any job in mcpu, synchronizing...
Use Luxid for querying cluster tags instead of SSConf
Not only there is no reason to use SSConf for that.Using it caused problems and failed tests as the query didn't get thelatest information due to distribution delays.
Bump serial numbers in ConfigWriter only in offline mode
Otherwise, WConfd takes care of it.
openvswitch fix
Support sequential evacuation
Make gnt-group evacuate support the --sequential option,which causes all evacuation moves to be execuated sequentially.This can be used to avoid congestion on a possibly slow linkbetween the node groups.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Add an option --sequential
...which can be used to tell commands like gnt-group evacuate tosequentially perform their action to keep load away from the cluster.
Merge branch 'stable-2.9' into stable-2.10
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
On expanding jobs, extend reason trail
Certain op-codes expand to a set of jobs. Forthose new jobs, extend their reason trail withthe reasons of the job that expanded to them.In this way, also for indirectly genreated jobsa complete trace back to the initiator can be...
Refactor instance create to accommodate unsafe OS scripts
This patch gives just the code structure. Implementation will follow.
Helper method to run OS scripts in a virtualized environment
This method temporarily changes the instance's disk to contain thehelper VM's boot disk and the target disk for the OS installation.
Helper method to handle the OS install package
This helper method either picks up the package passed through the OSparameters or creates a new package and updates the instance's OSparameters. This allows users to add their own OS install package.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Extract code that runs OS scripts to a separate method
... to make it simpler to introduced the install helper VM.
RPC 'os_export'
The 'os_export' RPC is used to export an OS to a compressed tarball.The path to this tarball will be given to the metadata daemon, whichwill then be used to serve this package to the instances via theinstance communication mechanism.
Generalize 'TemporaryDisk' to handle any number of disks
... including different disk templates, accesses, and sizes.
Move 'DetermineImageSize' to 'ganeti.cmdlib.common'
Move 'DetermineImageSize' from 'ganeti.cmdlib.backup' to'ganeti.cmdlib.common' so it can be used by disk zeroing and instanceinstallations. This patch also transforms this method into a functionand generalizes the error messages....
Add trusted/untrusted create scripts
Add timeouts for OS install helper VM
Add timeout parameters to client, opcodes, LU, etc. The startuptimeout is similar to that one used in zeroing, but the shutdowntimeout is different because it is a single timeout that includes thetime needed to install plus shutdown....
Add 'install_image' to bootstrap, CLI, opcodes, and queries
Add 'install_image' param to 'Cluster'
The 'Cluster.install_image' param holds the location of the image tobe used for the safe installation instances.
Fix 'IsInstanceRunning' to handle empty 'instance_info'
Instance info might return an empty dictionary in case the instancedoes not exist. Fix 'IsInstanceRunning' to handle that fact.
Fix var reference from 'instance' instead of 'lu.instance'
... given that 'lu.instance' does not exist in all LUs.
Fix dashes in environment names for OS params
Fix sorting in CLI options
Move instance communication network CLI option around.
Fix exception message to be consistent with others
Fix typo in documentation
Do not start/stop ganeti-masterd
This daemon does not do anything anymore, so wecan as well get rid of it.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>Signed-off-by: Klaus Aehlig <aehlig@google.com>
Filter disallowed compression tools
This patch makes import and export operations fail when anon-whitelisted compression type is used. The whitelisted tools have tobe specified manually via"gnt-cluster modify --custom-compression-tools=<tools>".
Signed-off-by: Hrvoje Ribicic <riba@google.com>...
Forbid the compression tools from being set over RAPI
Although we impose restrictions on what can be input as a compressiontool, someone with RAPI credentials could choose and execute a numberof commands by first setting the custom tools and then executing them....
Fix minor RAPI forbidden parameter handler error
The forbidden parameter functionality did not work if renames were notpresent due to a missing default. This patch fixes the issue.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Add the compression tools parameter
This patch makes the myriad of changes necessary for the compressiontool parameter to be added. The filtering of compression tools forsuspicious entries has been added for this exact purpose.
Make arbitrary compression tools work
We assume that the compression tools the user supplies use stdin andstdout for handling data, and that a switch is used to distinguishcompression from decompression. This patch introduces these constraintsby adding the invocation of these tools to the import-export daemon....
Correct docstrings
The previously present docstrings noted that only certain values wereto be used, and this patch correct this now that it is not trueanymore.
Disable protections against unknown compression types
Ganeti took care to restrict all possible compression invocations tothe few options that were available. This patch strips away all ofthose, but does not allow any interesting and dangerous commands...
Extend offered compression types
This patch adds a few new types of compression supported by Ganeti:gzip-fast (gzip -1), gzip-slow (ordinary gzip), and lzop. gzip nowbecomes a shorthand for gzip-fast kept for compatibility.
Replace DRBD reservations in config.py with WConfd calls
.. and remove unused code.
The verification of the configuration needs to be later included in theconfiguration verification checks in WConfd (see issue #759).
Clean up remove_locks
Locks are now identified by their name, and no longera resource in the master daemon. In particular, there isno more need for explicitly removing the locks; freeingthem is enough.
Fail in replace-disks if attaching disks fails
Previously, if attaching the new secondary during a replace-disksoperations failed, only a warning was emitted. The subsequent sync-disksoperation cannot finish in such a case, however.
Therefore, this patch changes the warning into an error. This way it's...
Conflicts: configure.ac # Taken both contributions
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
KVM: set IFF_ONE_QUEUE on created tap interfaces
The IFF_ONE_QUEUE flag directs the kernel to only queue tap packets once(as opposed to queueing them twice, once for the device, and once for theqdisc), possibly avoiding interface stalls when one of the queues overruns....
Simplify cleanup of locks
Since, from stable-2.12 onwards, locks are no longer explicitlyadded and removed, there is no need to release them separately.The freeing of all locks of the level left will take care of thisanyway.
Handle lock addition as lock acquisitions
From stable-2.12 onwards no longer are explicitly added; they justexist for all conceivable names/uuids. Nevertheless, addition washandled special based on the assumption that no one else can havea lock on an entity that is just being created....
Shorten debug log
...by leaving out the full state of the configuration. These overly longlines also make it hard to find the log entry needed.
Fix the broken unittests
Fix the unittest that have been broken when we lifted the Disk objectsfrom the Instances.
Signed-off-by: Ilias Tsitsimpis <iliastsi@grnet.gr>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Lift the Disk objects from the Instances
This patch replaces 'instance.disks' with 'GetInstanceDisks' everywherein the codebase. From now on, the function 'GetInstanceDisks' from theconfig file has to be used in order to get the disks of an instance.Also the functions 'AddInstanceDisk'/'RemoveInstanceDisk' have to be...
Add 'instance_disks' parameter to 'CreateDisks'
Modify 'CreateDisks' function to accept the disks of an instance as anextra parameter. Since the instance may not have been saved to theconfig file yet, 'CreateDisks' can not query the config file for the...
Add 'disks_info' Instance slot
Instance's 'disks' slot will contain a list of disk UUIDs. Create a newslot named 'disks_info' which will be annotated with the disk objects'during the RPC.
Fix a 'map' application in ImageDisks
In the first branch of 'if' the 'disks' variable is of type'list of tuples' whereas in the second branch ('else') it isof type 'list of disk objects'. Fix the second one to agreewith the first.
Signed-off-by: Ilias Tsitsimpis <iliastsi@grnet.gr>...
Remove a disk from an instance in config
Implement functions 'RemoveDisk' and 'DetachInstanceDisk'. The first oneremoves a disk from the config file and the second one detaches a diskfrom an instance. A wrapper 'RemoveInstanceDisk' is provided in order to...
Add a disk to an instance in config
Implement functions 'AddDisk' and 'AttachInstanceDisk'. The first oneadds a new disk to the config file and the second one attaches a disk toan instance. A wrapper 'AddInstanceDisk' is provided in order to add adisk and attach it to an instance at once (because Ganeti doesn't...
Add methods to config to get disks
'GetInstanceDisks' returns a list of disk objects for the giveninstance. 'GetDiskInfo' returns information about a disk given its UUID.These functions should be used instead of the Instance's disk method.
Also add the 'getDisk' and 'getInstDisks' functions in Haskell but leave...
Get MapLVsByNode from config
Use 'GetInstanceLVsByNode' from config file instead of Instance'sMapLVsByNode method.
Also remove all_lvs computation from config's 'AddInstance' method. Inorder to compute the lvs we have to use the 'GetInstanceLVsByNode'...
Get instance's nodes from config
Use 'GetInstanceNodes' to get the nodes of an instance, instead of the'all_nodes' Instance property.
Get instance secondary nodes from config
Use 'GetInstanceSecondaryNodes' to get the secondary nodes of aninstance, instead of the 'secondary_nodes'.Annotate 'secondary_nodes' in rpc so that backend has access to it.
Override disks in BuildInstanceHookEnvByObject
The function BuildHooksEnv in LUInstanceRemove, calls the functionBuildInstanceHookEnvByObject which needs to query the config to getthe secondary_nodes/disks of an instance. The function BuildHooksEnvwill be called before and after the Exec function. The second time...
Add 'GetInstanceLVsByNode' to config
Provide a mapping of node to LVs a given instance owns.
Add 'GetInstanceNodes' to config
Get all disk-related nodes for an instance.Also use 'GetInstanceSecondaryNodes' to get thelist of secondary nodes.
Add timestamp/serial_no slot to disk objects
Now that disks are top level citizens in config,they need a timestamp and a serial_no slot.
Add 'all_nodes' property to disk objects
This property returns the nodes covered by a disk.
Add disks entry to config.data
Add disks entry to config.data.
Fix a list expression to conform to our style guide
This fixes an error occurring in 'make commit-check'.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
kvm: use a dedicated QMP socket for kvmd
The KVM daemon keeps a persistent connection to the instances' QMPsockets, listening for asynchronous events. As each monitor socket(either human, or QMP) can handle only one client at a time, this hasthe side-effect that QMP cannot be used for regular instance operations....
Conflicts: Makefile.am doc/design-draft.rst lib/bootstrap.pyResolution:...
Set correct Ganeti version on setup commands
When asked to execute a setup command, prefix it witha command sequence to test for the existence of theneeded Ganeti version and switching to it.
Add a utility to combine shell commands
Add a function that combines individual shell commands intoa single command (calling the standard shell) that executesthe given commands in sequence while they succeed.
noded: start metad via daemon-util
Signed-off-by: Apollon Oikonomopoulos <apoikos@gmail.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
KVM: start kvmd via daemon-util
Fix the order of configuration calls
Since _SetDrbdHelper calls methods on self.cfg, it's necessary to updatethe configuration before it is called.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Fix missing comma
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Merge branch 'stable-2.11' into master
lint
Pass the debug level to forked jobs
When forking off jobs, make them inherit the debug levelof the parent process (i.e., of luxid). In this way, wecan debug jobs in test clusters without cluttering productionlogs. We pass the debug level through the environment instead...
Make jobs not log as master-daemon
While jobs are still handled by what used to be masterd,the log message should be assigned to the respectivejob.