Fix remote imports
A simple typo…
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
http.client: Disable SSL session ID cache
This patch disables the SSL session ID cache for all cURL operations.This is needed because http.HttpBase's PyOpenSSL implementation does notcurrently set a context using SSL_set_session_id_context(3SSL), cURLtries to re-use the session ID and, according to...
Merge branch 'stable-2.2' into devel-2.2
Set list of trusted SSL CAs for client to verify
As per SSL_CTX_set_client_CA_list(3SSL), set the list of acceptable CAsadvertised to SSL clients to include the server's own certificate. Thisevidently fixes the pycurl/gnutls RPC client.
During the TLS Handshake, when client verification is requested, the...
RPC: disable curl's Expect header
This patch solves the very slow (~8-9 seconds) gnt-instance modifybehaviour. Well, it solves in general the slow RPC behaviour, but it wasmost visible in that LU.
It seems that curl's behaviour with regard to file uploads (via PUT) and...
Export VG name via LUQueryConfigValues
This will be used by LUXI client programs to display the VG name.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
LUGetTags: Acquire locks in shared mode
Retrieving tags can be done while the lock is shared. Only writingneeds to be exclusive.
Also add a FIXME for cluster tags, where the code currently doesn'tuse any locks except the config lock.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
LUDelTags: Improve formatting of error message
Use utils.CommaJoin to add spaces after comma, clean up code a bit.
Before: Tag(s) 'bar','baz','foo','moo' not foundAfter: Tag(s) 'bar', 'baz', 'foo', 'moo' not found
cli: Move parsing of --net option to separate function
This function will also be used in tools/move-instance.
kvm: collapse two consecutive extend calls
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
kvm: Introduce support for -mem-path
Using hugepages, KVM instances can get a good performance boost. Toactivate that, we need to pass the -mem-path argument to KVM along withthe mount point of the hugetlbfs file system on the node.
For the sake of memory availability computation, we use the -mem-prealloc...
Rename the _oss cluster vars to _os
Per the mailing list discussion, rename _oss to _os, both in cluster parametersand in the rest of the code.
This is just an s/_oss/_os, with the exception of a small bit of cleanuparound the helper_os function in cmdlib.py....
KVM: Add function to check the hypervisor version
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix instance rename regression from 3fe11ba3
Committ 3fe11ba3 broke the instance rename as we don't use the FQDNanymore. This fixes it.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Sort OS names and variants in LUDiagnoseOS
The OS list and variants as returned from LUDiagnoseOS is not sorted,and gnt-instance reinstall doesn't sort it either. This means that itthe menu that users are present with is inconsistent across clusters,and that is confusing....
Change behaviour of OpDiagnoseOS w.r.t. 'valid'
This patch changes the behaviour of OpDiagnoseOS with regards to the'valid' field to be similar to the one for the hidden/blacklistedfields: unless this field is requested, invalid OSes are filtered out....
Allow gnt-os modify to change the new OS params
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add two more _T-type tests
These are useful for more in-depth checking of some kinds of arguments.
Add blacklisted/hidden OS support in LUDiagnoseOS
This changes the behaviour of LUDiagnoseOS significantly.
The addition of hidden/blacklisted OSes would mean that each user-facingclient would have to filter intentionally such OSes from display, whichis not a good choice. Rather, the patch makes LUDiagnoseOS not return...
Restrict blacklisted OSes in instance installation
Add two new cluster settings
The new variables are:
- a list of hidden OSes, that should not be displayed to the users in interactive selection (e.g. reinstall); however, if they are selected, they can be used- a list of OSes that should be hidden and blocked from install-time selection...
Abstract OS name/variant functions
Currently, the computation of the 'pure' name or the variant ishardcoded and spread around the functions that need it. This is notnice, and in the future we'd spread it even more with more usage ofvariants/pure os names....
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Merge branch 'devel-2.1' into devel-2.2
Fix mac checker regex
Currently, the mac checker regex could match a corner case of11:22:33:44:55:66: (one extra colon at the end). We fix this, and wealso move the regex compilation outside of this function, at modulelevel.
Signed-off-by: Iustin Pop <iustin@google.com>...
Fix migration on new KVMs
New KVMs (0.12.1.2-el6 and 0.13.5 tested) exit immediately afterunsuccessful network connection when they are in "-incoming" mode. Thesimple check netutils.TcpPing causes remote kvm to exit so the migrationwill always fail. This check is also redundant by the way as if the...
cli: Pass options in {Add,Remove}Tags
They'll be used for job priorities. Also add an empty line tognt-os where it's missing.
opcode summary: improve display for list summaries
Currently, opcodes like NODE_EVAC_STRATEGY look bad:
89684 error NODE_EVAC_STRATEGY([u'node3'])
With this patch, we try to render list arguments a little bit better:
89684 error NODE_EVAC_STRATEGY(node3)...
cmdlib: Fix type of “name” parameter for tag operations
The parameter “name” is be None for cluster tags.
rlib2: Set tag operation param “name” to None for cluster tags
Otherwise parameter verification in the master daemon fails.
Remove mcpu's ReportLocks callback
This is no longer needed with the new lock monitor. One callback is kept tocheck for cancelled jobs.
Revert "jqueue: Resume jobs from “waitlock” status"
This reverts commit 4008c8edae31a3971fa8c4b200238afc8005d3d4.
While it worked in my initial tests, I've now found cases where this doesn'twork properly as it is. More work is needed and will be done as part of the...
Fix OS_VARIANT variable setting
This was introduced in efaa9b06d1e1e6d1678d0edd75b1ba37cf0de3d9.
in OSCoreEnv: inst_os.name is pure operating system name (without variant) as variant is stripped in OSFromDisk(). So we always get variant = inst_os.supported_variants[0] (first...
Fix pylint warning in http/__init__.py
My bad for not seeing this before:R0201:614:HttpBase.GetSslCiphers: Method could be a function
Allow SSL ciphers to be overridden in HTTP server
Users of this class, such as the RAPI server, might want to override or adjustthe default SSL cipher defined in a constant.
jqueue: Resume jobs from “waitlock” status
After an unclean restart of ganeti-masterd, jobs in the “waitlock” status canbe safely restarted. They hadn't modified the cluster yet.
jqueue: Move queue inspection into separate function
This makes the init function a lot smaller while not changingfunctionality.
jqueue: Don't update file in MarkUnfinishedOps
This reduced the number of updates to the job files. It's used in two placeswhile processing a job and the file is updated just afterwards.
locking.SharedLock: Update class docstring
This was already outdated when the initial version of SharedLock was addedin commit 162c1c1f1 (February 2008).
cli: Use list of options shared between commands
The completion script for bash has to know about these options. Until nowthe list was in two places--once in cli.py and once inautotools/build-bash-completion. A shared list is used with this patch.
jqueue: Use separate function for encoding errors
Comes with unittest.
Fix some epydoc warnings
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix breakage introduced by commit 8044bf655
Note to self: even patches removing one line can break everything.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Manuel Franceschini <livewire@google.com>
Remove “dry_run” from opcodes.OpCreateInstance
I'ts declared twice, once in opcodes.OpCode and here, and this oneis redundant.
Show list of pending acquires in “gnt-debug locks”
This is accomplished by keeping a list of waiting threads insteadof just their number inside the lock-internal condition. A fewother tweaks to the output format are also made.
Adding a paramiko fingerprint format helper
And provide unittests for them
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add simple lock monitor
This patch adds an initial implementation of a lock monitor, accessiblefor the user through “gnt-debug locks”. It currently shows all resourcelocks: BGL, nodes and instances. Config and job queue locks could beshown too, but wouldn't be of much help. The current owner(s) and mode...
workerpool: Allow setting task name
With this patch, the task name is added to the thread name and will show up inlogs. Log messages from jobs will look like “pid=578/JobQueue14/Job13 mcpu:289DEBUG LU locks acquired/cluster/BGL/shared”.
Use one function to parse “--fields” option values
locking.LockSet: Use function to get member lock name
Fix small spelling mistake
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Stop adding the dry-run option by default
Currently cli.py unconditionally adds the dry-run option. This patchdisables this, and exports dry-run as a normal option.
The other alternative I tried to implement (adding a new fake option fordisabling the auto-add per individual command) would require changes in...
jqueue: Remove lock status field
With the job queue changes for Ganeti 2.2, watched and queried jobs areloaded directly from disk, rendering the in-memory “lock_status” fielduseless. Writing it to disk would be possible, but has a huge cost atruntime (when tested, processing 1'000 opcodes involved 4'000 additional...
Don't ignore secondary node silently
Currently on non-mirrored disk templates the secondary node is ignoredsilently. This patch adds a check for this case, and warns the usershould this be happening. This solves issue 113.
The patch also moves a prereq check to an argument check. This is ok...
Conflicts: man/gnt-instance.sgml...
RAPI client: Support modifying instances
RAPI: Allow modifying instance
gnt-debug: Extend job queue tests
Test multiple opcodes, also with failure.
jqueue: Mark opcodes following failed ones as failed, too
When an opcode fails, the job queue would leave following opcodes as “queued”,which can be quite confusing. With this patch, they're all marked as failed andassertions are added to check this.
jqueue: Work around race condition between job processing and archival
This is a simplified version of a patch I sent earlier to make sure the jobfile is only written once with a finalized status.
jqueue: More checks for cancelling queued job
We can also check when the lock status is updated. This willimprove job cancelling.
errors: Function to check whether value is encoded error
jqueue: Add more debug output
Merge branch 'devel-2.1'
Re-add the 'live' parameter to migration opcodes
This patch reintroduces the live parameter, for backwards compatibilityat the Luxi level. This way, clients can work transparently with both2.1 and 2.2, even though sub-optimally.
Restore 'tablet mouse on vnc' behavior
We needed to revert commit 5b062a58ac76b39c2dc6a7e1543affdf43dc7ee7because it was in conflict with the usb_mouse hv parameter. Here wereintroduce its functionality only when usb_mouse is not specified.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Revert "Add -usbdevice tablet to KVM when using vnc"
This reverts commit 5b062a58ac76b39c2dc6a7e1543affdf43dc7ee7.This fixes issue 109. The mouse type can be set with the usb_mousehv parameter.
workerpool: Don't keep reference to task arguments
The workerpool should not keep any reference to task arguments afterthey were processed by RunTask. Doing so led to jobs being cachedby the job queue's WeakValueDictionary for longer than they should've...
cli.SubmitOpCode: Pass keyword parameter as keyword
masterd.instance: Add missing argument
_DiskTransferPrivate takes three parameters, not two.
Adjust message in case ~/.ssh is no directory
Use actual path, not something hardcoded.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
RAPI client: Fix docstring for migrating instance
RAPI client: Support renaming instances
Allow renaming instances via RAPI
RAPI client: Don't re-use PycURL object
With this patch, a new PycURL object will be created for each request.This should make the RAPI client safe for simultaneous calls frommultiple threads. Unittests are adjusted accordingly.
An unnecessary variable assignment is also removed from the unittest...
Fix bug in bdev when drbd version format is x.x.x.x
This patch fixes a bug reported in [0]. Newer drbd versions can haveanother digit beyond the regular major, minor and point release digits.We modify the regex used to match that with an optional parts which is...
Allow instance NIC's IP address to be None
Also add some assertions.
Add new parameter type “maybe string”
Before strict checking was implemented, NIC IP addresses could be setto “None”. Commit bd061c35 added more strict checking, includingenforcing the IP address to be a string. With this new type, itcan again be set to None....
cmdlib: Change expected type for source CA on remote import
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix a few job archival issues
This patch fixes two issues with job archival. First, theLoadJobFromDisk can return 'None' for no-such-job, and we shouldn't addNone to the job list; we can't anyway, as this raises an exception:
node1# gnt-job archive foo...
Change handling of non-Ganeti errors in jqueue
Currently, if a job execution raises a Ganeti-specific error (i.e.subclass of GenericError), then we encode it as (error class, [errorargs]). This matches the RAPI documentation.
However, if we get a non-Ganeti error, then we encode it as simply...
Instance migration: remove error on missing link
Since we don't support upgrades from 1.2.4 without restarting theinstance, the 'not restarted since 1.2.5' check/error iswrong/misleading.
Since the live migration works anyway without the links (it recreates...
Add check for RAPI paths to start with /2
During a discussion in July 2010 it was decided that we'll stabilize on /2. Seemessage ID <20100716180012.GA9423@google.com> for reference.
RAPI client: The os argument for instance reinstalls is optional
Signed-off-by: David Knowles <dknowles@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
RAPI client: Support migrating instances
workerpool: Add two additional assertions
workerpool: Change signature of AddTask function to not use *args
By changing it to a normal parameter, which must be a sequence, we canstart using keyword parameters.
Before this patch all arguments to “AddTask(self, *args)” were passed asarguments to the worker's “RunTask” method. Priorities, which should be...
RAPI: Support migrating instances
workerpool: Move waiting for new tasks for a worker to the pool
This way fewer private variables of the pool are accesssed by the worker.
workerpool: Remove unused worker method
HasRunningTask is never used except for an assertion, where wedon't really need the lock.
workerpool: Additional check in BaseWorker.ShouldTerminate
Document that it should only be called from within RunTask andadd an assertion for this. This means we can no longer use amethod on the pool and hence remove WorkerPool.ShouldWorkerTerminate.
workerpool: Use common function to add tasks
Rename masterfailover to master-failover
Most (all?) of our commands use dash-separator: replace-disks,verify-disks, add-tags, etc. “gnt-cluster masterfailover” is an oldexception to this rule.
The patch replaces it with master-failover, add a compatiblity alias,...
RAPI: Add os params to instance creation v1
Since the RAPI QA suite doesn't seem to offer easy testing of failedcreations, I didn't add this to the QA. Pointers on how to do it arewelcome.
The patch also changes the 'os' argument to be required, since that is...
Improve handling of lost jobs
Currently, if the cli.JobExecutor class is being used, and one of thejobs is being archived before it can check its result, it will raise astracktrace as _ChooseJob is not prepared to handle this case.
This case makes JobExecutor work better with lost jobs (it still reports...
luxi: convert permission errors into exception
This patch adds handling of permission errors so that we don't showtracebacks when a non-root user runs a gnt-* command. Since in thefuture we'll have different permissions, we need to handle this in RAPItoo....