Merge branch 'devel-2.4' into grnet-2.4-ippool-devel
Add UnescapeAndSplit unittest for multi-escapes
This would have caught the bug in the first place. Argh,hand-generated test cases!
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix a bug in command line option parsing code
Fix bug affecting command line options of "keyval" type. Althoughescaping commands with \ is supported, it is is not applied to theinput recursively.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>Signed-off-by: Iustin Pop <iustin@google.com>...
merge upstream devel-2.4
Conflicts: lib/cli.py lib/cmdlib.py lib/config.py lib/opcodes.py
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
ConfigWriter: Fix epydoc error
The parameter is called “mods”, not “modes”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
LUGroupAssignNodes: Fix node membership corruption
Note: This bug only manifests itself in Ganeti 2.5, but since theproblematic code also exists in 2.4, I decided to fix it there.
If a node was assigned to a new group using “gnt-group assign-nodes” the...
Ensure unused ports return to the free port pool
Ensure ports previously allocated by calling ConfigWriter's AllocatePort() arereturned to the pool of free ports when no longer needed:
Re-wrap a paragraph to eliminate a sphinx warning
This just makes sure that the paragraph doesn't contains lines thatstart with :, which make Sphinx (1.0.7) complain.
Update NEWS and increase to 2.4.5
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix queue archive creation with wrong permissions
On a master failover some of the archive dirs might have wrongpermissions in the non-root model. This is due to the nature of nodedstill running as root and the job queue is synced that way. This patchwill fix this behaviour by setting the permissions accordingly....
Update NEWS for unreleased 2.4.5
I need this for another 2.5 release.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
rpc: Disable HTTP client pool and reduce memory consumption
We noticed that “ganeti-masterd” can use large amounts of memory,especially on large clusters. Measurements showed a single PycURL clientusing about 500 kB of heap memory (the actual usage depends on versions,...
Fix assertion error on unclean master shutdown
Commit 66bd7445 added an assertion to ensure a finalized job has its“end_timestamp” attribute set. Unfortunately it didn't cover a case whenthe queue is recovering from an unclean master shutdown.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
utils: Fix UnescapeAndSplit parsing bug
If a value passed to UnescapeAndSplit ended with a backslash anexception would be raised:
$ gnt-instance modify -H mem=x\\ inst1.example.com[…] e2 = slist.pop(0)IndexError: pop from empty list
Version bump 2.4.4
Update NEWS file
Documentation fix for importing with --src-dir option
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>(cherry picked from commit b7d7876bd0e9844fab8be28bfa1fd5d563ec7412)
Conflicts:
lib/cmdlib.py (easily fixed)
Adding missing test data for commit 7a380ddfc
Fix a parsing issue with DRBD 8.3.11 in the Linux Kernel
In the Linux kernel commit 4b0715f096 introduced a display bug into/proc/drbd which broke our regex.
The bug was first introduced into Linux 2.6.39-rc1. This bug is stillunfixed as of today.
This patch adapt the regular expression to workaround this bug for the...
ensure-dirs: Fix a bug with queue/archive permissions
While it sets the permission on all files in queue/archive accordinglyit doesn't do so for the created archive directories. This patch fixesthis problem.
Signed-off-by: René Nussbaumer <rn@google.com>...
Fix typo in NEWS
doc/admin: s/grub/GRUB/
“GRUB” is an acronym for GRand Unified Bootloader.
Bumping version to 2.4.3
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Update the NEWS file for 2.4.3
Fix small typo in docstring
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
“--dry-run” starts with two dashes.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Remove 15-second sleep from LUInstanceCreate
Remove 15 second sleep when wait_for_sync is not set. LUInstanceCreate alreadycalls _WaitForSync with oneshot=True, which already performs an internalwait-loop for disks to start syncing.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Pause DRBD sync for OS install if no_wait_for_sync
When no_wait_for_sync is specified in LUInstanceCreate, Ganeti lets DRBD syncin the background while performing the rest of the installation steps,including OS installation.
However, OS installation is a very disk-intensive task that intereferes badly...
Add a flag to burnin to allow specifying VCPU count.
Signed-off-by: Pedro Macedo <pmacedo@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add support for cluster/OS parameters in QA
Currently there is no way to QA with (for example) an initrd becausethe QA only inits the cluster with the default parameters. This makesit impossible to QA using anything but the default parameters, whichdoesn't always work....
Add OS search path to gnt-cluster info
Otherwise, it's pretty hard to figure it out from the command line.
Signed-off-by: Ben Lipton <benlipton@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Reopen daemon's stdio on SIGHUP
Before this patch daemons would continue to refer to an old logfile fortheir standard I/O if they had been asked to reopen the log (SIGHUP).
Reopen log file only once after SIGHUP
Commit b6fa9a44 added a re-openable log handler. The log file isreopened when a daemon is sent a HUP signal. Due to a bug in the code,fixed by this patch, the log file would be reopened for every single logmessage thereafter....
Don't leak file descriptors when setting up daemon output
When a daemon's output is configured using “utils.SetupDaemonFDs”, thefunction must use dup2(2). Unfortunately the code didn't close theoriginal file descriptors, leaking them in the process.
Fix aliases in bash completion
Ever since commit 2d48a3a2 aliases were not included in the bashcompletion script. This patch also replaces one tab with two spaces.
gnt-node volumes: Fix instance names
Commit 84d7e26b changed “objects.Instance.MapLVsByN” to not just returnthe LV name, but to include the volume group name (e.g.“xenvg/d67e8700….disk0_data”). This in turn broke the mapping of volumenames in LUNodeQueryvols, stopping instance names from displayed in...
rlib2: handle wait_for_sync in instance creation
Until now, wait_for_sync was harcoded to True when creating instances via RAPI.We allow settings wait_for_sync via RAPI and have rlib2 honour this decision.
Note: this is not needed in the upcoming 2.5, as Opcode parameters are not...
KVM: strip boot=on from IDE CDROM drivers
boot=on forces KVM to use an extboot ROM which has shown poor behaviour withbooting from CD images. IDE CDROM emulation (which is forced by default whenbooting from CD) does not need boot=on, so we simply omit it....
ht: Add new check for numbers
Places which receive floats can usually also deal with integers, e.g.OpTestDelay. Tests are added and the new check function is used for theaforementioned opcode and verifying query results.
Fix off-by-one bug in job serial generation
Commit 009e73d0 (September 2009) changed the job queue to generatemultiple job serials at once. Ever since it would return one more thanrequested.
The “serial” file in the job queue directory is defined to contain the...
Shorten some unbreakable lines in man pages
In order to make the display right on 80-columns terminals.
Correct some spelling mistakes
New lintian is even smarter:
- overriden → overridden- allows to → allows one to
RAPI: handle tags during instance creation
Add tag handling to _ParseInstanceCreateRequestVersion1.
Fix bug in recreate-disks for DRBD instances
The new functionality in 2.4.2 for recreate-disks to change nodes isbroken for DRBD instances: it simply changes the nodes without caringfor the DRBD minors mapping, which will lead to conflicts in non-empty...
Fix a lint warning
Patch db8e5f1c removed the use of feedback_fn, hence pylint warnnow.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
KVM: configure bridged NICs at migration start
Commit 5d9bfd870 moved tap interface handling from KVM to Ganeti, partlyto also solve the problem of routed interfaces getting configured tooearly during live migrations, causing network anomalies. In that...
Fix RAPI documentation regarding master role
Fix bug in drbd8 replace disks on current nodes
Currently the drbd8 replace-disks on the same node (i.e. -p or -s) hasa bug in that it does modify the instance disk temporarily beforechanging it back to the same value. However, we don't need to, andshouldn't do that: what this operation do is simply change the LVM...
LUInstanceCreate: use opcodes.RequireFileStorage
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Don't add ",boot=on" to disks on kvm >= 0.14
Under newer kvm this prevents the vm from starting.Ah, change!
KVM: fix per-instance stored UID value
When using the pool security model, ExecuteKVMRuntime was storing theinstance's UID using str(uid), which would result in storing theLockedUid._repr__() result:
$ cat /var/run/ganeti/kvm-hypervisor/uid/xxxxxxxxxxxxx...
Fix issue with tags on instance creation
Commit 720f56c85a added the ability to specify tags when creating aninstance. The “tags” attribute of an instance object needs to be a set,but the patch's code saved it as a list, causing breakage in other parts...
Network query: handle default NIC link value
When asked to return statistics about a network usage by instances, we didn'tproperly handle the link=None case. This patch fixes the behaviour by fallinkback to the cluster default link.
Update network design doc
Add network and IP pool config methods and ssconf
Add helper methods to ganeti.config.ConfigWriter to handle Network addition andremoval, IP address reservation and commitment to the address pools.
Also add the “networks” ssconf key to make network names/UUIDs available to all...
query: add network query primitives
Add NetworkQueryData data container for network-related queries. Also add anetwork query field generator and helper functions to generate usage statisticsfor IP address pools.
bootstrap: initialize network configuration to an empty dict
Add pool management unit tests
Add unit tests to check proper IP address pool management logic.The tests cover all simple operations on an IP address pool.
Add network-related LUs
Add Logical Units for network adding and querying network config objects.
LUGroupSetParams: handle network map/unmap
Modify LUGroupSetParams to accept network addition/removal. Networks areconnected to nodegroups' links.
Add network-related opcodes
Add opcodes for network addition and query.
Add network query LUXI method
Add gnt-network client script
gnt-network manipulates network objects. Required parameters have been added toganeti.cli.
Instance addition and removal: manage IP addresses
Prototype implementation of IP address management during instanceaddition/removal.
Document network-related hooks
Network management design doc
Make bash completion network-aware
Make zsh completion network-aware
Fix cyclic dependency between iallocator and pools
The IP pool management infrastructure currently needs to know an instance'snode to fill in IP addresses. For this to happen, the iallocator must runbefore we generate IP addresses, yet the iallocator itself needs the instance's...
Fix typo in gnt-network list
ToStdOut -> ToStdout
More fixes regarding ip pool mgmt and instance creation
Add doc/examples/zsh_completion to .gitignore
Improve temporary reservation of IP addresses
Add a new method, TemporaryReservationManager.GetECReserved(), which returns allreservations held by a given ec_id.
Also add a new method, ConfigWriter._UnlockedCommitReservedIps(), which can becalled to commit all temporarily reserved IP addresses to the respective...
ConfigWriter.ReleaseIp: change semantics
Change the semantics of ReleaseIp, so that it can be called from outsidewithout knowing the exact network UUID, by passing only node and link.
RemoveInstance: handle IP default NIC link
Call cluster.SimpleFillNIC to get the default link when link is None.
IP pool management during instance modification
Perform suitable pool operations on NIC addition, removal and modification.
Add tag handling to {Op,LU}InstanceCreate
Add a tag slot to opcodes.OpInstanceCreate. We do not reuse _PTags, as this isintended for OpTagsSet and thus:
a) is not documented b) does not carry a default value, making it mandatory
Also pass the tags to the iallocator during instance creation....
Add tagging option to gnt-instance create
Add TAG_ADD_OPT option to cli.py and use it in gnt-instance. Modifycli.GenericInstanceCreate() accordingly.
Export instance tags to instance hooks
Instance hooks now get an INSTANCE_TAGS environment variable, which contains aspace-delimited list of the affected instance's tags.
Also update the documentation to reflect the change.
TemporaryReservationManager: decrement retry counter
The retry counter used in TemporaryReservationManager.Generate() was notdecremented for each failed retry, as presumably intended. This patchdecrements the retry counter after each failed attempt.
Handle StopIteration in free IP generation
Catch the StopIteration exception and return None to the temporary_ipsreservation manager, so that it fails at the 64-attempt boundary.
Handle address pool exhaustion properly in cmdlib
We catch the ConfigurationError raised by ConfigWriter.GenerateIp() inLUInstanceCreate and LUInstanceSetParams, so that we can raise an OpPrereqexception and better inform the user about the error.
gnt-network info: include external reservations
Also include the list of externally reserved IP addresses in an address pool inthe output of gnt-network info <network_name>.
Add lib/ippool.py: IP pool management primitives
This patch introduces primitive structures for IP pool management. The core ofthe IP Pool management framework is the ippool.IPv4Network class, which encodesan IPv4 subnet configuration with its IP address pool. The pool functionality...
Add config objects and methods for IP pools
This patch introduces the following changes to lib.config and lib.objects tofacilitate IP pool management.
- Add a "networks" key to objects.cluster to hold link -> subnet relations. Each link is assumed to be associated with at most one IPv4 and at most one...
LUCreateInstance: Retrieve IPs from an IP pool
A new ip keyword is added ("pool") to signify that LUCreateInstance should getthe instance's IP from an IP pool (rather than manually or by DNS resolution).
IP and link checks are re-ordered so that a NIC's link is available at the time...
lib.objects: Add network object definition
Add a network object definition, with the appropriate slots, to lib.objects.
Also add a “networks” top-level configuration slot to host network definitionsand a “networks” slot to the NodeGroup objects to hold network ←→ node-group...
Add ganeti.network: address pool management tools
lib.network contains AddressPool, a helper class for ganeti.objects.Network.AddressPool wraps a Network object and provides all IPv4 Address Poolmanagement logic, manipulating the respective fields of the Network object....
Commit 5d9bfd870 moved tap interface handling from KVM to Ganeti, partly toalso solve the problem of routed interfaces getting configured too early duringlive migrations, causing network anomalies. In that direction, configuration of...
Add one forgotten element to the file disk path
This was left out during the fix/refactoring
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
LUInstanceCreate: fix file storage dir calculation
- Move the calculation at the beginning of CheckPrereq, since it doesn't modify any state, but still keeps locks- Only perform the calculation if the actual disk template is filebased- Error out if there is no defined file storage dir...
Check that filestorage is enabled when requested
Remove self.op.file_storage_dir isabs check
As the manpage says, and the code does, self.op.file_storage_dir is anadditional relative path under the cluster file storage dir. As such itshould not be absolute.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Remove duplicate CDROM_IMAGE_PATH KVM hv param def
Introduced during merge, because of fba7f91. Remove the second definition, asit broke KVM HTTP CDROM boot.
jqueue: Fix potential race condition when cancelling queued jobs
When a job was cancelled, its status would be changed and the filewritten again. Since this was a final status, the job file could bemoved anytime for archival. If the job was still in the queue, however,...
Fix argument order in ReserveLV and ReserveMAC
ConfigWriter.ReserveLV() and Configwriter.ReserveMAC() calledTemporaryReservationManager.Reserve() with the ec_id and resource argumentsswapped. As a result, two reservation attempts for the same resource type...
TLReplaceDisks: Move assertion checking locks
Commit 1bee66f3 added assertions for ensuring only the necessary locksare kept while replacing disks. One of them makes sure locks have beenreleased during the operation. Unfortunately the commit added the check...
node evac: don't call IAllocator if no instances
Currently we generate an empty list only for the '-n node' invocation,but for iallocator we still call the iallocator (which needs an RPCcall, etc.). By moving the computation of instances outside of the if...
RPC/Backend: Make UploadFile uid and gid agnostic