Dimitris Aragiorgis [Tue, 6 Nov 2012 18:18:05 +0000 (20:18 +0200)]
Merge branch 'stable-2.6-hotplug' into stable-2.6-ippool-hotplug-esi
Dimitris Aragiorgis [Tue, 6 Nov 2012 18:10:06 +0000 (20:10 +0200)]
Hotplug: do not allow disk/0 to be hotplug-able
BIOS boots from first disk. If we enable hotplug in disk/0
it will added at the end of PCI address space (slot 31). If
another disk gets added then it will be added just before and
BIOS will try to boot from an non bootable device and the
instance will hung.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Sun, 4 Nov 2012 22:29:25 +0000 (00:29 +0200)]
Merge branch 'stable-2.6' into stable-2.6-ippool-hotplug-esi
Conflicts:
lib/cmdlib.py
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Michael Hanselmann [Thu, 1 Nov 2012 15:04:33 +0000 (16:04 +0100)]
Fix typo in gnt-instance man page
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 1 Nov 2012 13:54:03 +0000 (14:54 +0100)]
jqueue: Return jobs to queue when shutting down
When a job is still waiting for locks and the queue is shutting down,
they should be returned and not actually start processing. Until now
jobs which transitioned from “queued” to “waiting” were already
considered to be running as far as the shutdown code was concerned.
This fixes issue 296.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 1 Nov 2012 14:23:54 +0000 (15:23 +0100)]
gnt-debug delay: Add "--submit" option
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Wed, 31 Oct 2012 16:10:19 +0000 (17:10 +0100)]
Make hostname checks uniform between instance rename and add
Currently, we have instance rename doing extra checks on the host
name, to prevent accidental wrong renames; however, instance create
doesn't do these checks (issue 291), which (if DNS is misconfigured)
can lead to hard to diagnose errors.
This patch abstracts the name checking from LUInstanceRename into a
separate function, which is then reused in both instance rename and
instance create.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Iustin Pop [Wed, 31 Oct 2012 13:47:18 +0000 (14:47 +0100)]
Improve logging of new job submissions
This addresses issue 290: when receiving new jobs, logging is
incomplete, and we don't have the job ID(s) and/or summaries
logged. Only later, when the job is queried for or being processed, we
know more.
This is not good when troubleshooting, so let's improve the initial
logging.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Iustin Pop [Wed, 31 Oct 2012 13:31:48 +0000 (14:31 +0100)]
Improve handling of lock exceptions
There are two issues with lock exceptions right now:
- first, we don't log the original error; this is fine for now
(locking.py always returns the same error here), but in general is
brittle: if locking.py would start returning more information, we'd
completely miss that
- second, an actual honest lock conflict is not an internal error;
it's simply an optimistic lock failing, and as such we should not
return internal error, but rather resource_not_unique
This addresses issue 287.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Iustin Pop [Tue, 30 Oct 2012 15:49:33 +0000 (16:49 +0100)]
Add note about developing on a production machine
This is the bit of documentation missing for issue 170. Doing
development on a machine which already has Ganeti installed kind of
works, but only when the installed and the developed version are very
similar, and even then it can be problematic.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Dimitris Aragiorgis [Tue, 30 Oct 2012 18:26:18 +0000 (20:26 +0200)]
Merge branch 'stable-2.6-ippool' into stable-2.6-ippool-hotplug-esi
Dimitris Aragiorgis [Tue, 30 Oct 2012 18:24:52 +0000 (20:24 +0200)]
Fix a bug intruduced when refactoring GenerateFree
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Tue, 30 Oct 2012 17:55:05 +0000 (19:55 +0200)]
Merge branch 'stable-2.6-ippool' into stable-2.6-ippool-hotplug-esi
Dimitris Aragiorgis [Tue, 30 Oct 2012 17:50:06 +0000 (19:50 +0200)]
Fix a typo in OpNetworkSetParams
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Tue, 30 Oct 2012 17:41:45 +0000 (19:41 +0200)]
Merge branch 'stable-2.6-ippool' into stable-2.6-ippool-hotplug-esi
Conflicts:
lib/network.py
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Tue, 30 Oct 2012 17:20:23 +0000 (19:20 +0200)]
Introduce new basic type TStringNone
Check if a certain value is "none". This is used by opcodes that
unset an object's attribute (set it to None), for instance
OpNetworkSetParams.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Iustin Pop [Tue, 30 Oct 2012 13:43:09 +0000 (14:43 +0100)]
Fix runtime memory increases
Commit
2c0af7da which added the runtime memory changes functionality
had a small typo (wrong name); I've rewritten this to only compute the
delta once, for simplicity.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Dimitris Aragiorgis [Tue, 30 Oct 2012 11:00:57 +0000 (13:00 +0200)]
Use bitarray 0.8 interface
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Mon, 29 Oct 2012 19:00:06 +0000 (21:00 +0200)]
Simplify GenerateFree in network module
GenerateFree now returns the first available IP in the network
or raises AddressPoolError if it is full.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Mon, 29 Oct 2012 18:39:58 +0000 (20:39 +0200)]
Replace string values with proper constants
_UnlockedCommitIp is used either for releasing or reserving an
IP inside a network. New constants RELEASE_ACTION/RESERVE_ACTION
are used to decide which is the case.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Mon, 29 Oct 2012 18:10:04 +0000 (20:10 +0200)]
Simplify mac generation funtions
Introduce new config method _UnlockedGetNetworkMACPrefix and refactor
original _GenerateOneMAC so that it takes prefix as an optional argument
and returns a function that actually generates a MAC. If prefix is not
given or None it uses the cluster level default.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Tue, 30 Oct 2012 12:57:48 +0000 (14:57 +0200)]
Use bitarray 0.8 interface
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Iustin Pop [Tue, 30 Oct 2012 11:53:26 +0000 (12:53 +0100)]
Fix validation of vgname in OpClusterSetParams
This variable can be empty, when we want to disable LVM, so we can't
use TMaybeString.
Fixes issue 285.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Tue, 30 Oct 2012 11:29:16 +0000 (12:29 +0100)]
Fix removal of storage directory on shared file storage
This patch makes _RemoveDisks symmetric to _CreateDisks with respect
to file-based storage: _CreateDisks uses "in constants.DTS_FILEBASED",
whereas _RemoveDisks was not update and only uses "==
constants.DT_FILE". This results in stale directories left on the
filesystem.
Fixes issue 262.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Tue, 30 Oct 2012 11:14:37 +0000 (12:14 +0100)]
Switch non-redundant check to disk template-based
Currently, the warning/notice about non-redundant instances in cluster
verify is based non empty secondaries list (how old is this?); the
proper way to check this nowadays is via DTS_MIRRORED.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Mon, 29 Oct 2012 12:01:58 +0000 (13:01 +0100)]
Document the new --yes-do-it option for master-failover
Sorry, this should have gone in the previous commit, I forgot about
it.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Iustin Pop [Mon, 29 Oct 2012 11:21:59 +0000 (12:21 +0100)]
Add option to force master-failover without voting
This fixes issue 282.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Iustin Pop [Fri, 26 Oct 2012 06:41:07 +0000 (08:41 +0200)]
Update instance modify message
Currently the message does not say explicitly that instance-initiated
reboots are useless to trigger the use of new parameters, per the
thread on the user mailing list. Let's improve it a bit.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Dimitris Aragiorgis [Fri, 26 Oct 2012 07:22:10 +0000 (10:22 +0300)]
Merge branch 'stable-2.6-hotplug' into stable-2.6-ippool-hotplug-esi
Conflicts:
lib/config.py
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Thu, 25 Oct 2012 19:17:56 +0000 (22:17 +0300)]
Hotplug: rpc only if instance is running
Intruduce new function _InstanceRunning().
Fix *PCIInfo config methods to hold exclusively _config_lock and
take instance object as an argument.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Simon Deziel [Fri, 11 May 2012 16:00:24 +0000 (12:00 -0400)]
Force tap's MAC prefix to "fe"
Setting a high prefix discourages the bridge from adopting the tap's
MAC. Xen is not affected by this since the MAC is forced to
"fe:ff:ff:ff:ff:ff".
This addresses issue #217.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit
be3f52f304767a719d4b0dc07c81a1aaea8432b0)
(this is critical enough that we want it on stable-2.6)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Dimitris Aragiorgis [Wed, 24 Oct 2012 13:30:29 +0000 (16:30 +0300)]
Merge branch 'stable-2.6-ippool' into stable-2.6-ippool-hotplug-esi
Dimitris Aragiorgis [Wed, 24 Oct 2012 13:27:49 +0000 (16:27 +0300)]
Fix a bug in LUNetworkAdd concering needed_locks
Fix network client to pass conflicts_check to the opcode.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Iustin Pop [Fri, 19 Oct 2012 13:19:48 +0000 (15:19 +0200)]
Fix disk adoption interaction with ipolicy checks
In Ganeti 2.6, disk adoption is broken due to the ipolicy checks being
done before we read volume size from remote nodes. We fix this by
simply moving these checks to after the disk adoption code which
updates the disk size; it's not that nice that we fail a (almost)
config-level check after we've reserved the LVs, etc., but we need to
do so in order to validate the ipolicy correctly.
Tested:
- normal instance creation
- creation via adoption with good size (pass)
- creation via adoption with wrong LV size (fail as expected)
- QA in progress
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Dimitris Aragiorgis [Fri, 19 Oct 2012 12:38:33 +0000 (15:38 +0300)]
Merge branch 'stable-2.6' into stable-2.6-ippool-hotplug-esi
Bernardo Dal Seno [Tue, 16 Oct 2012 13:01:48 +0000 (15:01 +0200)]
Better NEWS file for 2.6.1
Better formatting of text, past tense used when appropriate.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Dimitris Aragiorgis [Tue, 16 Oct 2012 13:54:34 +0000 (16:54 +0300)]
Fix OP_RESULT for OP_NETWORK_QUERY
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Tue, 16 Oct 2012 12:53:36 +0000 (15:53 +0300)]
Allow passing 'none' for network_type option
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Tue, 16 Oct 2012 11:11:20 +0000 (14:11 +0300)]
Merge branch 'stable-2.6-ippool' into stable-2.6-ippool-hotplug-esi
Dimitris Aragiorgis [Tue, 16 Oct 2012 11:01:10 +0000 (14:01 +0300)]
Fixes to pass unittests (make check)
Conflicts:
doc/rapi.rst
lib/ovf.py
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Tue, 16 Oct 2012 08:59:52 +0000 (11:59 +0300)]
Merge branch 'stable-2.6-hotplug' into stable-2.6-ippool-hotplug-esi
Conflicts:
lib/cmdlib.py
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Tue, 16 Oct 2012 08:57:18 +0000 (11:57 +0300)]
Merge branch 'stable-2.6-ippool' into stable-2.6-ippool-hotplug-esi
Dimitris Aragiorgis [Tue, 16 Oct 2012 06:04:37 +0000 (09:04 +0300)]
Fixes to pass pylint (make lint)
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Tue, 16 Oct 2012 06:00:46 +0000 (09:00 +0300)]
Fix _GetPCIInfo() to pass unittests
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Tue, 16 Oct 2012 05:56:07 +0000 (08:56 +0300)]
Add HOTPLUG_OPT in gnt-backup import client
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Tue, 16 Oct 2012 05:51:54 +0000 (08:51 +0300)]
Export nic.network during gnt-backup export
Modify FinalizeExport() in backend.py to export nic.network as well.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Mon, 15 Oct 2012 18:19:28 +0000 (21:19 +0300)]
Fix CIDR opcode checks to accept 'none' value
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Mon, 15 Oct 2012 14:50:00 +0000 (17:50 +0300)]
Export serial_no of networks/instances in hooks
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Mon, 15 Oct 2012 14:25:04 +0000 (17:25 +0300)]
Export serial_no and uuid in network queries
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Mon, 15 Oct 2012 13:54:31 +0000 (16:54 +0300)]
Fix locks in network LUs
Acquire corresponding instance locks if conflicts_check is True.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Bernardo Dal Seno [Fri, 12 Oct 2012 15:28:03 +0000 (17:28 +0200)]
Update NEWS and bump version to 2.6.1
This is a small bug-fix only release.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Thu, 11 Oct 2012 05:21:07 +0000 (07:21 +0200)]
verify-disks: Explicitely state nothing has to be done
Example output:
$ gnt-cluster verify-disks
Submitted jobs 4327
Waiting for job 4327 ...
No disks need to be activated.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 10 Oct 2012 09:44:13 +0000 (11:44 +0200)]
Add list of design documents implemented in version 2.6
Each version should have its dedicated list.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Bernardo Dal Seno [Fri, 28 Sep 2012 19:22:22 +0000 (21:22 +0200)]
Better list of replace-disks arguments + typos fixed
The man page and the bultin-in help for gnt-instance replace-disks were
inconsistent. Also fixed some typos in man pages.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Fri, 5 Oct 2012 02:36:42 +0000 (04:36 +0200)]
jqueue: Look at archived jobs when watching
First: This enables the use of “gnt-job watch $id” for archived jobs.
Now, the reason for actually making this work is that during
sufficiently large group or node evacuations jobs are archived before
the client gets to poll for their output. This led to situations where
the jobs would finish successfully, but the client reported an error
because it couldn't see the job anymore.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
(cherry picked from commit
045694691691af1b10d1d055bdac55cf205e47de)
Michael Hanselmann [Wed, 3 Oct 2012 00:15:55 +0000 (02:15 +0200)]
Show old primary/secondary node on disk replacement
People unfamiliar with Ganeti's internals might be confused with the
different hostnames showing up later in the process.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Dimitris Aragiorgis [Wed, 3 Oct 2012 15:35:42 +0000 (18:35 +0300)]
Check for existing networks/groups in Network LUs
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Michael Hanselmann [Wed, 3 Oct 2012 01:41:11 +0000 (03:41 +0200)]
gnt-instance reinstall: Don't always exit with success
If one or more jobs failed the exit status should be set accordingly.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Tue, 2 Oct 2012 18:41:19 +0000 (20:41 +0200)]
LUClusterVerify: Ignore /proc/drbd if DRBD is disabled
This fixes issue 190. The problem was that the check for DRBD was
enabled if LVM storage is used and didn't depend at all on whether DRBD
is enabled.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit
3d8ae32740475f35ac2e7a4937f5469700202e5b)
Gintautas Miliauskas [Sun, 30 Sep 2012 18:44:21 +0000 (20:44 +0200)]
Fixed typos in devnotes.rst
Signed-off-by: Gintautas Miliauskas <gintas@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Bernardo Dal Seno [Thu, 13 Sep 2012 20:25:32 +0000 (22:25 +0200)]
Always_failover doesn't require --allow-failover anymore
If an administrator sets always_failover, it means that there is no need
for another explicit approval to failover instead of migrating.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit
b5f0b5cc16dd3221df06e2fc1353b048eef17d06)
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 24 Sep 2012 13:03:53 +0000 (15:03 +0200)]
bash_completion: Enable extglob while parsing file
In older versions of GNU Bash extended patterns, such as “@(…)”, are only
available with the “extglob” shell option. As pointed out in [1] and [2],
“extglob” must be enabled while parsing the code. Therefore the flag must be
enabled at the beginning of the script and be reset to its original value at
the end as to not interfere with other code on shell initialization.
[1] http://unix.stackexchange.com/questions/45957
[2] http://mywiki.wooledge.org/glob
Reported by Sascha Lucas.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit
893ad76d18e1a8286247e06c8451bbd4831cff8f)
Constantinos Venetsanopoulos [Fri, 21 Sep 2012 09:32:07 +0000 (12:32 +0300)]
Merge branch 'stable-2.6-esi' into stable-2.6-ippool-hotplug-esi
Constantinos Venetsanopoulos [Fri, 21 Sep 2012 09:24:24 +0000 (12:24 +0300)]
Check if extstorage's log dir exists
Check if extstorage's log dir exists on the host machine, before
providing a logfile name. We need to be sure the dir exists,
otherwise RunCmd will fail ugly when passed a logfile that cannot
be opened.
Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Constantinos Venetsanopoulos [Thu, 20 Sep 2012 15:29:13 +0000 (18:29 +0300)]
Do not show logfile on ExtStorage's attach error
During attach the logfile is `None'. If the attach script
fails and we try to Tailfile() its logfile, TailFile
breaks with:
TypeError: coercing to Unicode: need string or buffer, NoneType found
Thus, we don't show the logfile, if ExtStorage's attach script fails.
Just throw the appropriate error.
Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Dimitris Aragiorgis [Wed, 12 Sep 2012 13:21:02 +0000 (16:21 +0300)]
Merge remote branch 'google/stable-2.6' into stable-2.6-ippool-hotplug-esi
Conflicts:
lib/rpc.py
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Michael Hanselmann [Wed, 12 Sep 2012 12:41:24 +0000 (14:41 +0200)]
rpc: Remove duplicated logic, fix unittests
Commit
5fce6a89 changed RpcRunner._InstDict to add the disk parameters
on all encoded instances. It didn't remove a special case in
“_InstDictOspDp”. Update and fix unittests as well.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Constantinos Venetsanopoulos [Wed, 12 Sep 2012 11:27:16 +0000 (14:27 +0300)]
Annotate disk params on instance_start
We call _GatherAndLinkBlockDevs during the process, which in turn
calls _RecursiveFindBD. This needs disk parameters to work.
See also commit b8291e0.
This was reported by Ansgar and Damien.
Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Wed, 12 Sep 2012 10:38:03 +0000 (12:38 +0200)]
cmdlib: Handle locking.ALL_SET correctly when copying locks
When locks are copied “locking.ALL_SET” must be handled separately
(ALL_SET has the value None). Reported by Constantinos Venetsanopoulos
who saw failover for RDB-based instances not working.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Thu, 6 Sep 2012 16:08:43 +0000 (01:08 +0900)]
Fix bug in non-mirrored instance allocation
The function `allocateOnSingle' has a bug in the calculation of the
cluster score used for deciding which of the many target nodes to use
in placing the instance: it uses the original node list for the score
calculation.
Due to this, since the original node list is the same for all target
nodes, it means that basically `allocateOnSingle' returns the same
score, no matter the target node, and hence the choosing of the node
is arbitrary, instead of being done on the basis of the algorithm.
This has gone uncaught until reported because the unittests only test
1 allocation at a time on an empty cluster, and do not check the
consistency of the score. I'll send separate patches on the master
branch for adding more checks to prevent this in the future.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>
Dimitris Aragriorgs [Wed, 5 Sep 2012 17:02:50 +0000 (20:02 +0300)]
Do not allow networks with less than 8 hosts
This is only needed in case base64 and bitarray.tostring() is used for
saving reservations. If bitarray.to01() was used then we would not have
this restriction.
Signed-off-by: Dimitris Aragriorgs <dimara@grnet.gr>
René Nussbaumer [Tue, 28 Aug 2012 11:14:58 +0000 (13:14 +0200)]
Fix gnt-debug iallocator
There was an issue with the recent ipolicy introduction which lead to a
bug in gnt-debug iallocator. It was not providing the spindle_use field
and therefore it wont let you create a valid iallocator request.
Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Tue, 28 Aug 2012 23:12:19 +0000 (01:12 +0200)]
Fix warnings/errors with newer pylint
To help developing Ganeti on newer distributions, let's try to fix
pylint warnings/errors. I'm using pylint from current Debian wheezy:
pylint 0.25.1, astng 0.23.1, common 0.58.0, and we have 3 things that
needs fixing.
First, a really wide "except", with the silencing in the wrong
place. I'm not sure why this doesn't have "except Exception", so let's
add it. However, pylint still complains about "Catching too general
exception", even though we do want to catch both system and our
exception, so let's add a silence for W0703. It's true that we
shouldn't catch KeyboardInterrupt and friends, but that should be
cleaned up on the master branch.
Second, pylint complains about "redefining name builtin tuple",
because we do some pattern matching in the except blocks in
netutils. This seems to be a false positive, but let's clean the code
around this.
And finally, type inference again goes bad, so let's silence E1103
with its "boolean doesn't have 'get' method".
After this, I can run "make lint", and by extension "make
commit-check" on Debian Wheezy, yay! We might be able to bump our
required pylint versions to something not ancient…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 28 Aug 2012 22:59:00 +0000 (00:59 +0200)]
Fix decorator uses which crash newer pylint
Pylint version:
pylint 0.25.1,
astng 0.23.1, common 0.58.0
crashes when passing the fully-qualified decorator name with:
File "/usr/lib/pymodules/python2.7/pylint/checkers/base.py", line 161, in visit_function
if not redefined_by_decorator(node):
File "/usr/lib/pymodules/python2.7/pylint/checkers/base.py", line 116, in redefined_by_decorator
decorator.expr.name == node.name):
AttributeError: 'Getattr' object has no attribute 'name'
I found out that simply using a shortened name will 'fix' this issue,
so let's do this to allow running newer pylint versions.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Dimitris Aragriorgs [Thu, 30 Aug 2012 19:31:37 +0000 (22:31 +0300)]
Refactor _GetPCIInfo() in cmdlib
Signed-off-by: Dimitris Aragriorgs <dimara@grnet.gr>
Dimitris Aragriorgs [Thu, 30 Aug 2012 19:31:37 +0000 (22:31 +0300)]
Refactor _GetPCIInfo() in cmdlib
Signed-off-by: Dimitris Aragriorgs <dimara@grnet.gr>
Dimitris Aragriorgs [Thu, 30 Aug 2012 18:43:48 +0000 (21:43 +0300)]
Add logging in case of hotplug
Signed-off-by: Dimitris Aragriorgs <dimara@grnet.gr>
Dimitris Aragriorgs [Thu, 30 Aug 2012 18:43:48 +0000 (21:43 +0300)]
Add logging in case of hotplug
Signed-off-by: Dimitris Aragriorgs <dimara@grnet.gr>
Dimitris Aragriorgs [Thu, 30 Aug 2012 17:40:25 +0000 (20:40 +0300)]
Maintain backwards compatibility in kvm runtime
Introduce _GenerateKVMBlockDevicesOptions()
In case hotplug is enabled per instance then the runtime file
contains one more field: block_devices. In this case kvm_cmd
is extended with block device options during _ExecuteKVMRuntime().
If no hotplug is enabled then runtime file has block_devices info
encapsulated in kvm_cmd and block device options are exteded
during _GenerateKVMRuntime.
Signed-off-by: Dimitris Aragriorgs <dimara@grnet.gr>
Dimitris Aragriorgs [Thu, 30 Aug 2012 17:40:25 +0000 (20:40 +0300)]
Maintain backwards compatibility in kvm runtime
Introduce _GenerateKVMBlockDevicesOptions()
In case hotplug is enabled per instance then the runtime file
contains one more field: block_devices. In this case kvm_cmd
is extended with block device options during _ExecuteKVMRuntime().
If no hotplug is enabled then runtime file has block_devices info
encapsulated in kvm_cmd and block device options are exteded
during _GenerateKVMRuntime.
Signed-off-by: Dimitris Aragriorgs <dimara@grnet.gr>
Iustin Pop [Mon, 27 Aug 2012 13:45:24 +0000 (15:45 +0200)]
Make stable-2.6 compatible with newer pep8
This is done so that all current branches can run with newer pep8;
note that instead of fixing the problems (like I did on master), I've
just silenced more. These should *not* be merged onto master!
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Dimitris Aragiorgis [Sat, 11 Aug 2012 09:21:50 +0000 (12:21 +0300)]
cfupgrade for removing oldstyle hotplug info
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Thu, 9 Aug 2012 13:10:52 +0000 (16:10 +0300)]
Use base64 format in network reservations slots
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Wed, 22 Aug 2012 15:08:10 +0000 (18:08 +0300)]
Merge branch 'stable-esi'
Conflicts:
Makefile.am
lib/cli.py
lib/client/gnt_instance.py
lib/cmdlib.py
lib/constants.py
lib/opcodes.py
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Constantinos Venetsanopoulos [Fri, 30 Mar 2012 08:41:37 +0000 (11:41 +0300)]
Add external storage interface related man pages
* ganeti-extstorage-interface man page
* gnt-storage man page
Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Constantinos Venetsanopoulos [Thu, 26 Jul 2012 13:41:08 +0000 (16:41 +0300)]
Initial ExtStorage support in htools
Constantinos Venetsanopoulos [Wed, 28 Mar 2012 09:32:56 +0000 (12:32 +0300)]
Add the gnt-storage client
Add a new client called 'gnt-storage'.
The client interacts with the ExtStorage interface, similarly to
the way gnt-os interacts with the OS interface.
For now, only two commands are supported: 'info' and 'diagnose'.
'diagnose' calculates the node status of each provider on each node,
similarly to gnt-os diagnose. Furthermore, for every provider, it
calculates it's nodegroup validity for each nodegroup. This is done
inside the LU and not the client (marked as 'TODO' for the global
validity of gnt-os diagnose).
In the future, gnt-storage can be used to manage storage pools,
or even be extended to diagnose other storage types supported by
Ganeti, such as lvm, drbd (INT_MIRROR) or rbd (EXT_MIRROR).
Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Constantinos Venetsanopoulos [Fri, 20 Jul 2012 15:38:10 +0000 (18:38 +0300)]
Add --allow-arbit-params to gnt-instance modify
Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Constantinos Venetsanopoulos [Wed, 28 Mar 2012 09:29:22 +0000 (12:29 +0300)]
Multiple ExtStorage Providers and ext-params
Add support for passing parameters to the ext template (ext-params).
Take advantage of disk-params, that don't seem to make much sense in
this template (ExtStorage Providers are not predefined and we don't
know their needs) and use them to pass the ext-params dynamically to
the template.
ext-params are correlated with gnt-os-interface's os-params.
All ext-params are exported to the ExtStorage Provider through it's
environment, with variables prefixed with 'EXTP_' (similarly to the
OS interface's 'OSP_' params).
ext-params are passed through the --disk option. If the disk template
is of type `ext' during instance add, then any additional options that
are not in IDISK_PARAMS given to --disk are considered ext-params
e.g.:
gnt-instance add -t ext --disk=0:size=2G,param1=value1,param2=value2
Finally, we introduce a new IDISK_PARAM called IDISK_PROVIDER, that is
mandatory for template `ext' and is used to select the desired
ExtStorage Provider. This parameter is not valid for other template
types.
The IDISK_PROVIDER parameter becomes the first element of the
disk's unique_id tuple e.g.:
unique_id = ('sample_provider1', 'UUID.ext.diskX')
Example selecting different ExtStorage Providers for each disk and
passing different ext-params to them:
-t ext --disk=0:size=2G,provider=sample_provider1,param1=value1
--disk=1:size=3G,provider=sample_provider2,param2=value2
Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Constantinos Venetsanopoulos [Mon, 12 Mar 2012 15:49:18 +0000 (17:49 +0200)]
Implement the External Storage Interface
With this commit we introduce the External Storage Interface
to Ganeti, abbreviated: ExtStorage Interface.
The ExtStorage Interface provides Ganeti with the ability to interact
with externally connected shared storage pools, visible by all
VM-capable nodes. This means that Ganeti is able to handle VM disks
that reside inside a NAS/SAN or any distributed block storage provider.
The ExtStorage Interface provides a clear API, heavily inspired by the
gnt-os-interface API, that can be used by storage vendors or sysadmins
to write simple ExtStorage Providers (correlated to gnt-os-interface's
OS Definitions). Those Providers will glue externally attached shared
storage with Ganeti, without the need of preprovisioned block devices
on Ganeti VM-capable nodes as confined be the current `blockdev' disk
template.
To do so, we implement a new disk template called `ext' (of type
DTS_EXT_MIRROR) that passes control to externally provided scripts
(the ExtStorage Provider) for the template's basic functions:
create / attach / detach / remove / grow
The scripts reside under ES_SEARCH_PATH (correlated to OS_SEARCH_PATH)
and only one ExtStorage Provider is supported called `ext'.
The disk's logical id is the tuple ('ext', UUID.ext.diskX), where UUID
is generated as in disk template `plain' and X is the disk's index.
Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Constantinos Venetsanopoulos [Mon, 2 Apr 2012 17:05:51 +0000 (20:05 +0300)]
Update the shared storage design document
Update the shared storage design document to reflect the current
changes, after the implementation of the ExtStorage interface.
Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Dimitris Aragiorgis [Wed, 22 Aug 2012 14:22:14 +0000 (17:22 +0300)]
Merge branch 'stable-hotplug'
Conflicts:
lib/cli.py
lib/client/gnt_instance.py
lib/cmdlib.py
lib/objects.py
lib/opcodes.py
lib/rpc.py
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Thu, 9 Aug 2012 11:04:10 +0000 (14:04 +0300)]
Hotplug: client support
Add --hotplug option.
Only used by OpInstanceCreate()/OpInstanceSetParams.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Thu, 9 Aug 2012 11:00:13 +0000 (14:00 +0300)]
Hotplug: cmdlib support
Hotplugging is done by functions invoked by ApplyContainerMods(). In
order hotplugging to take place a --hotplug option must be passed.
Hotplug can enabled for a VM if --hotplug is passed during
gnt-instance add (hotplug_info will be added).
In order initial devices of a VM to be hotpluggable hotplug_info must be
early available. So _GetPCIInfo() gets either lu.hotplug_info in case of
LUInstanceCreate() or invokes lu.cfg.GetPCIInfo() in case of
LUInstanceSetParams(). To return a pci slot into the pool
lu.cfg.UpdatePCIInfo() must be invoked.
If no --hotplug is passed during gnt-instance modify --net add the nic
won't get a pci value and will be placed automaticaly in a free pci slot
after reboot. This means that is not going to be hotpluggable for the
rest of the instance's life.
To keep pci pool consistent we do not let a device that has been
hotplugged (i.e. has pci value) to be removed without --hotplug.
Before hotplugging a Disk it must be assembled. During
LuInstanceSetParams() newly created disks are not added to the instance
so _ExpandCheckDisks() in _AssembleInstanceDisks() will fail. So we make
this check optional only for this case.
In order to remove a disk (with blockdev_remove) it must be shutdown. So
after unplugging the disk _ShutdownDiskInstanceDisks() must be invoked.
Everything else is just like nic hotplugging.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Thu, 9 Aug 2012 10:59:11 +0000 (13:59 +0300)]
Hoplug: rpc support
Introduce new RPCs that eventually invoke hypervisor specific
hotplug functions.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Thu, 9 Aug 2012 10:55:01 +0000 (13:55 +0300)]
Hotplug: KVM support
KVM >= 1.0 is needed.
Introduce new methods: HotAddNic/HotDelNic
Both qemu monitor interface for hotplugging. If a device has
hotplug info (idx, pci) then:
- it gets named after it (e.g. virtio-net-pci.%d % idx ).
- it gets placed to the pci slot (hex(pci)) of bus 0.
If no hotplug info exist or kvm version is old then no hotplug takes
place and modifications will take place after reboot:
- devices will get traditionaly named
- and placed automaticaly in empty pci slots
NIC hotplug:
- open a tap and get its file descriptor.
- pass fd with SCM rights (using python-fdsend) via monitor socket
- create netdev and device with proper qemu interface
Disk hotplug:
- create drive with drive%d % idx name
- create device virtio-blk-pci.%d % idx
In order to migrate a VM that has devices hotplugged, an identical
VM should be booted with -incoming. To this end every time a hotplug
takes place runtime file must be updated.
For disk hotplug support we removeds disk from kvm_cmd in
runtime file add added them separately just like nics. So code building
disk related options has been moved out from _GenerateKVMRuntime()
and added to _ExecuteKVMRuntime().
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Thu, 9 Aug 2012 10:54:18 +0000 (13:54 +0300)]
Hoplug: objects modifications/additions
Introduce new HotplugInfo object as part of Instance object.
hotplug_info { 'nics': 0, 'disks': 0, 'pci_pool': [16, 17, 18] }
nics/disks is an index used for device identification and naming.
Everytime a new device (disk/nic) is hotadded these values get incremented.
pci_pool is a list of pci slots where a device can be placed. Every
time a device gets hot-plugged/unplugged a pci slot gets popped/appended.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Fri, 27 Jul 2012 07:50:04 +0000 (10:50 +0300)]
Add tags in network objects
Support:
- gnt-network add --tags.
- gnt-network list-tags/add-tags/remove-tags/.
- gnt-network list -o +tags
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Mon, 18 Jun 2012 16:05:28 +0000 (19:05 +0300)]
Change default fields for network listing
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Dimitris Aragiorgis [Thu, 7 Jun 2012 15:51:34 +0000 (18:51 +0300)]
Pass detailed network info in hooks
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>