cmdlib: Add opcode to modify storage unit fields
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add RPC calls to modify storage fields
storage: Add function to modify fields
This allows the “allocatable” flag on LVM PVs to be changed.
Merge commit 'origin/branch-2.1' into feature/containers
Add automated disk repair changes to design doc
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add review script
I've been using this script for a while to update commits beforepushing them to the main repository. It copies all commits in arange to another branch using git cherry-pick and starts an editorto modify the Reviewed-by: line(s) for each commit. The script is...
Implement “gnt-node physical-volumes” command
This command can be used to list all physical volumes on nodes.
Add new opcode to list physical volumes
storage: Use constants.py instead of local constants
storage: Fix semantics for directory size
The actual directory size is "used" space, not the total space onthe filesystem.
Merge branch 'next' into branch-2.1
Add “gnt-job watch” command
This command can be used to follow the output of a job. It's usefultogether with the --submit parameter for other commands.
jqueue: Fix error when WaitForJobChange gets invalid ID
When JobQueue.WaitForJobChange gets an invalid or no longer existing job ID ittries to return job_info and log_entries, both of which aren't defined yet.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
jqueue: Update message for cancelling running job
cmdlib: Change tasklet logging to debug level
rapi: Add /2/nodes/[node_name]/migrate resource
gnt-node: Use new opcode to migrate node
cmdlib: Add new opcode to migrate node
It migrates all primary instances from the node to their secondaries.
rapi: Add default parameter to _checkIntVariable
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
cmdlib: Add logging for tasklets
cmdlib: Fix tasklets handling if no tasklets are added
If no tasklets are added, self.tasklets evaluates to None. The LU baseclass will throw an exception because it thinks the derived class doesn'timplement the right methods.
rapi: Add /2/[node_name]/evacuate resource
This can be used to evacuate a node.
Add information about storage units framework
This updates the 2.1 design document with storage units framework information.
Signed-off-by: Iustin Pop <iustin@google.com>Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add RPC calls for storage unit list
Add first implementation of generic storage unit framework
utils: Add functions to calc directory size and free space on filesystem
These will be used by the new storage unit framework.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Build HTML from Ganeti 2.1 design
Collapse SSL key checking/overriding for daemons
Signed-off-by: Guido Trotter <ultrotter@google.com>
Collapse daemon's main function
With three ganeti daemons, and one or two more coming, the daemon's mainfunction started becoming too much cut&pasted code. Collapsing most ofit in a daemon.GenericMain function. Some more code could be collapsedbetween the two http-based daemons, but since the new daemons won't be...
Slightly abstract the daemon logfile lookup
The original LOG_<DAEMON_NAME> constants for daemon logfiles are gone.In their place there is a DAEMONS_LOGFILES dict, indexed by daemon name.
This is a minor change with the objective to uniform most of thedaemon's main() functions code, which is very similar one to the other....
Remove <DAEMON>_PID constants
The <DAEMON>_PID constants were created to reference a daemon pid file,but actually contain a daemon's name, because the various functions thatwork with pidfiles abstract the filename from the daemon namethemselves. Removing the constants and using the actual daemon name...
Move rapi to GetDaemonPort
Currently rapi is the only daemon which accepts a port option, ratherthan querying its own port from services, and failing back to thedefault if not found. Changing this to conform to what other daemons do.
Also update the ganeti-rapi(8) manpage...
Change GetNodeDaemonPort to GetDaemonPort in utils
GetNodeDaemonPort is used to lookup the node daemon port in the servicesfile, and if not found to return the default one. We make it a genericfunction, which accepts the daemon name in input, so that it can be used...
lvmstrap: Change diskinfo to use GenerateTable
This way the produced table is formatted nicely.
Signed-off-by: Stephen Shirley <diamond@google.com>Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Get rid of constants.RAPI_ENABLE
This constant is unused, except in qa. Removing it since it's always True.
This patch also removes the unused qa_rapi.PrintRemoteAPIWarningfunction, and removes a comment about temporary constants "until we havecluster parameters"....
cmdlib: Add init to Tasklet class
Remove references to utils.debug
Various modules set it to True when called in debugging mode, but theutils module supports no such global.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
ganeti-rapi, replace hardcoded exit value
substitute exit(1) with exit(constants.EXIT_FAILURE).Also fix a wrongly indented line.
Add the bind-address option to ganeti-rapi
cmdlib: Move LUMigrateInstance functionality to tasklet
gnt-node: Use new opcode to evacuate nodes
Add new opcode to evacuate nodes
cmdlib: Convert _DiskReplacer to tasklet
cmdlib: Function to get all secondary instances on a certain node
noded: Abstract hard-coded sys.exit value
On machines without the ssl file noded exists '5'.Changing this to constants.EXIT_NOTCLUSTER.
Also utils.GetNodeDaemonPort hasn't risen errors.ConfigurationError fora while, so removing that try/except block....
cmdlib: Add tasklet support to logical unit base class
cmdlib: Add tasklet base class
Add an example "ethers" hook
This hook can be used to update /etc/ethers with instance's macaddresses. A dhcp server on the nodes can then serve to the instancestheir correct address. (This has been tested with dnsmasq's dhcpimplementation)
ganeti-confd design doc
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
burnin: move batch init/commit into a decorator
Many burnin steps initialize the batch queue at the beginning and commitit at the end of their operation. This patch moves this code to adecorator, in order to reduce redundant code.
Signed-off-by: Iustin Pop <iustin@google.com>...
burnin: move instance alive checks to a decorator
Many burn steps to a manual check of instance aliveness, via duplicatecode. This patch moves this code to a decorator.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
burnin: Implement retryable operations
Some burnin steps are idempotent: e.g. reinstalling an instance (fromburning p.o.v.) can be done multiple times without any side-effects thatwould affect later burnin steps. As such, failing the whole burninprocess due a reinstall failure is undesirable....
Generate a shared HMAC key at cluster init time
This key is shared on all nodes (via cmdlib._RedistributeAncillaryFiles)and will be used for HMAC authentication of confd messages.
Fix unittests broken by commit 2bb5c9115f
File "../test/ganeti.hooks_unittest.py", line 239, in setUp self.lu = FakeLU(FakeProc(), self.op, self.context, None)File "…/ganeti/cmdlib.py", line 92, in init self.LogStep = processor.LogStepAttributeError: FakeProc instance has no attribute 'LogStep'...
cmdlib: Move code doing disk replacements into separate class
This class will be used for a new opcode to evacuate nodes.
cmdlib: Pass config and rpc objects directly to IAllocator
Before IAllocator would access them using “self.lu.cfg” and “self.lu.rpc”.It shouldn't know about the internals of the LU.
Ignore vim swap files
Fix backend import errors from GetHypervisorClass
The merge of commit 360b0dc into branch-2.1 broke import of backend,since it uses hypervisor.GetHypervisor() which returns an instance ofthe hypervisor. Some of the hypervisors create directories at init time,...
burnin: fix removal errors hiding real errors
A long-standing bug in burnin makes errors during the removal phase(e.g. because an import has failed, or because the initial creation hasfailed) hide the original error.
This patch suppresses removal errors if we are already in ‘has_err’...
Conflicts: lib/backend.py: non-trivial conflict but easy to solve
backend: Only build once the list of upload files
The list of upload files is built currently at every UploadFile() call.This patch moves it to a separate variable which is initialized onlyonce.
This won't make much difference but I regard it as cleanup....
Merge commit 'origin/next' into branch-2.1
Conflicts: lib/cli.py: trivial extra empty line
Fix gnt-instance reinstall
Commit 55efe6dabe48e5c37dc1ff6099e0bb8afde7a468 "Convert instancereinstall to multi instance model" actually broke instance reinstall forsingle-instance cases. This one-liner fixes it.
Fix a couple of epydoc warnings
It seems epydoc needs fully-qualified references, and doesn't deal withrelative ones (not even in the current module) if there are anyambiguities.
There are other epydoc warnings, in the rapi docstrings, but those areleft as-is as they're removed in 2.1....
job queue: fix loss of finalized opcode result
Currently, unclean master daemon shutdown overwrites all of a job'sopcode status and result with error/None. This is incorrect, since theany already finished opcode(s) should have their status and resultpreserved, and only not-yet-processed opcodes should be marked as...
Switch gnt-debug submit-job to JobExecutor
Currently gnt-debug submits jobs individually, but in 2.1 JobExecutoruses the optimized SubmitManyJobs luxi call and as such should be usedwhenever multiple jobs need to be submitted.
This patch converts gnt-debug submit-job to use it and also removes an...
gnt-instance batch-create: use the job executor
This small patch changed the batch create functionality to use the jobexecutor instead of single-job submits.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>...
Convert instance reinstall to multi instance model
This patch converts ‘gnt-instance reinstall’ from single-instance tomulti-instance model; since this is dangerours, it's required to pass“--force --force-multiple” to skip the confirmation.
Modify cli.JobExecutor to use SubmitManyJobs
This patch changes the generic "multiple job executor" to use the manyjobs submit model, which automatically makes all its users use the newmodel.
This makes, for example, startup/shutdown of a full cluster much more...
Add a luxi call for multi-job submit
As a workaround for the job submit timeouts that we have, this patchadds a new luxi call for multi-job submit; the advantage is that all thejobs are added in the queue and only after the workers can startprocessing them....
job queue: fix interrupted job processing
If a job with more than one opcodes is being processed, and the masterdaemon crashes between two opcodes, we have the first N opcodes markedsuccessful, and the rest marked as queued. This means that the overall...
Fix an error path in job queue worker's RunTask
In case the job fails, we try to set the job's run_op_idx to -1.However, this is a wrong variable, which wasn't detected until theslots addition. The correct variable is run_op_index.
Add slots on objects in jqueue
Adding slots to _QueuedOpCode decreases memory usage (of these objects)by roughly four times. It is a lesser change for _QueuedJobs.
ganeti.initd: Pass $*_ARGS to programs when restarting them
Optimizie OpCode loading
This patch converts the opcode loading to a pre-built map (at importtime) instead of iteration over the globals dict at each call.
Microbenchmarks show that this should be around three times faster, andburnin still passes.
Yet another fallout from the pylint fixes
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
Merge branch 'master' into next
Fix another issue with hypervisor_name change
Update NEWS and version for 2.0.2 release
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Improve the description of node flags in man page
[iustin@google.com: slightly reworded the explanation for offline andchanged the commit message]Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add enabled hypervisors to TestConfigRunner
This parameter is now mandatory for the cluster config to work.
Add a few more checks to verify config
- Check that the enabled hypervisors list is valid- Check that the master node is a valid node
Make sure enabled_hypervisors list is valid
Get rid of the default_hypervisor slot
Currently we have both a default_hypervisor and an enabled_hypervisorslist. The former is only settable at cluster init time, while the lattercan be changed with cluster modify.
This becomes cumbersome in a few ways: at cluster init time for example...
design-2.1: Update OS Flavours section
This reflects a discussion we had, according to which the full"parameters" implementation is too heavy weight for 2.1, and we shouldhave a partial version for now, and decide again later.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Change default stripe count to 1
In order not to change the default during a stable series, we modifyconfigure.ac to default to one stripe, in effect keeping the status quo(well, minus the LVM Attach() changes).
cmdlib: Use dict.fromkeys instead of custom loop
Simplify InitConfig and remove SimpleConfigWriter
InitConfig currently creates the cluster config_data, then puts it intoa dict, passes it to SimpleConfigWriter to load it from a dict (whichjust reuses the dict value) and then saves it. The SimpleConfigWriter is...
InitCluster, don't use SimpleConfigWriter
InitConfig returns a SimpleConfigWriter to InitCluster, which thenpasses it on to ssh.WriteKnownHostsFile, which extracts a couple ofvalues from it. One line later the full ConfigWriter is initialized.
By initializing it one line before we can pass the full writer to...
Fix python 2.4 compatibility
I got overexcited and forgot we have to remain compatible with python2.4. With this patch we move from sha256 to sha1 for hmac authenticatedserialized messages, and we handle both newer and older python, byimporting the right module for each....
Use full-stripe size in LVM growth
LVM has issues when growing stripped volumes, so it's best to specifythe growth in exact multiples of the full stripe size (as precise aspossible). For this we need to do a couple of changes: - in LVM Attach(), we query additionally the VG extent size and the LV...
Remove ConfigWriter.InitConfig
It's been replaced by a simpler bootstrap.InitConfig function, whichdoes the same job, and is currently unused.
Conflicts:
daemons/ganeti-masterd...
Remove SimpleConfigWriter.SetMasterNode
This function is not used.
_GenerateDiskTemplate: use base_index in the name
Currently if a disk is added later the base_index is not considered, andall the disks are called disk0. This patch fixes it.