Change bootstrap.SetupDaemonNode to use scp as we can assume SSH is setup
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'devel-2.2'
hansmi helped me with merging the conflict. Thanks
Conflicts: lib/workerpool.py
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Adding host key verification to setup-ssh
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Adding a paramiko fingerprint format helper
And provide unittests for them
IPv6 design doc part
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'devel-2.1' into devel-2.2
Conflicts: NEWS - merge configure.ac - keep the 2.2 version
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Increase version to 2.1.7
Update NEWS for the 2.1.7 release
workerpool: Add support for task priority
To add job priorities, the worker pool underlying the job queue mustsupport priorities per task. This patch adds them to the worker pool.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix race condition in locking unittest
While writing unittests for the new lock monitor, I made a small mistake anddidn't synchronize acquiring locks properly. This patch takes care of thisissue by adding additional synchronization.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Add simple lock monitor
This patch adds an initial implementation of a lock monitor, accessiblefor the user through “gnt-debug locks”. It currently shows all resourcelocks: BGL, nodes and instances. Config and job queue locks could beshown too, but wouldn't be of much help. The current owner(s) and mode...
workerpool: Allow setting task name
With this patch, the task name is added to the thread name and will show up inlogs. Log messages from jobs will look like “pid=578/JobQueue14/Job13 mcpu:289DEBUG LU locks acquired/cluster/BGL/shared”.
Use one function to parse “--fields” option values
locking.LockSet: Use function to get member lock name
Finalize removal of “lock_status” job field
When removing the field in commit 9bdab62198, I forgot to remove some parts.
Prepare 2.2.0~rc1 release
Update NEWS and version number.
Switch to the RPC call to update /etc/hosts in LUAddNode and LURemoveNode
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add RPC calls to update /etc/hosts
Fix a few epydoc docstrings
Make family argument in FormatAddress optional
By doing this we delegate the task of finding the correct address familyto the FormatAddress method.
Support IPv6 for instances
setup-ssh: fix updating of authorized_keys
Due to what seems like a bug (or inconsistency) in paramiko, filesopened with a+ over SFTP need a seek() in order for the user to be ableto read data from them. We implement this, and rely on the fact that wedo iterate over all lines before writing and that the file is opened in...
setup-ssh: Also use keys from the ssh-agent
Currently, setup-ssh only uses one disk-based key. This means that anysetup where we use keys from ssh-agent (which do not necessarily existon disk) will break when moving from the old method to setup-ssh.
This patch moves the SSH key handling to separate functions, and uses...
setup-ssh: fix the logging error message
This took a bit of interesting debugging, but the issue seems to be thaton older python/logging/paramiko (I'm not sure which is at fault here),the other threads are finishing later than the main thread, so thelogging module closes the logging file (but doesn't unregister it)...
setup-ssh: only read the ssh port once
Then reuse it. With certain setups, this could be slow when pre-seedingan entire cluster.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
setup-ssh: redo the logging levels
There is not enough logged by default on stderr, and way too muchinformation in the log file.
Since we don't want to debug paramiko itself, we filter only warningsand above.
Otherwise, we make WARNINGS the default level, with verbose meaning INFO...
setup-ssh: try to use key auth first
This patch changes the setup-ssh workflow to try key authenticationfirst, and then fall-back to password authentication. The password isalso read lazily, with no prompts if we can authenticate via keys.
Signed-off-by: Iustin Pop <iustin@google.com>...
Use Sphinx' :rfc: extension to refer to RFCs
Document non-standard usage of JSON in RAPI
This was requested in issue 118.
gnt-node add: add error msg when using IPv6
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Don't provide --no-ssh-key-check for QA
My last setup-ssh patch series removed this option. Though, this isjust a temporary fix as a patch series is coming which revert thatbehaviour to the old state
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Use family in backend.StartMaster
This patches changes the StartMaster method to consult the clusterprimary ip version when deciding whether to use arping or ndisc6 afteractivating the master ip.
Signed-off-by: Manuel Franceschini <livewire@google.com>...
Make Hostname object always intialize its name to fqdn
This patch restores the behaviour of Hostname (previously known asHostInfo) to always use fqdn. This was broken due to the fact that thenow used getaddrinfo does not return an fqdn in contrast togethostbyname_ex()....
Fix small spelling mistake
Fix a few commands behaviour with dry-run
These commands use or display the result from the LU, so in case ofdry-run, they will crash or display just 'None'. At least checking thatthe result is 'true' (in the boolean sense) will make them work better....
Stop adding the dry-run option by default
Currently cli.py unconditionally adds the dry-run option. This patchdisables this, and exports dry-run as a normal option.
The other alternative I tried to implement (adding a new fake option fordisabling the auto-add per individual command) would require changes in...
Explicitly add dry-run to some commands
Based on manual inspection (that the command only does a submit of somejobs/opcodes), we re-add the dry-run option on a subset of the existingcommands.
A few more commands could use dry-run, but the code doesn't deal nicely...
jqueue: Remove lock status field
With the job queue changes for Ganeti 2.2, watched and queried jobs areloaded directly from disk, rendering the in-memory “lock_status” fielduseless. Writing it to disk would be possible, but has a huge cost atruntime (when tested, processing 1'000 opcodes involved 4'000 additional...
QA: Run simple job queue test
utils: Use WriteFile in {Set,Remove}EtcHostsEntry
This avoids duplicate effort and has been a TODO for a long time.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Don't ignore secondary node silently
Currently on non-mirrored disk templates the secondary node is ignoredsilently. This patch adds a check for this case, and warns the usershould this be happening. This solves issue 113.
The patch also moves a prereq check to an argument check. This is ok...
Document primary ip version and ndisc6
Modify gnt-node add to call external script
Make it possible to invoke RunCmd in interactive mode
This is needed so we can run external scripts asking for password and suchto run interactive. Downside is that we can't get the output of the programbut we can still use the exit code.
Signed-off-by: René Nussbaumer <rn@google.com>...
Removing all ssh setup code from the core
etags: force Python as a language
etags has cannot properly identify the scripts/gnt-* files as Python, sowe statically force the language to Python.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Support for IPv6 QA
Support IPv6 node add
Support IPv6 cluster init
Add primary_ip_family to ssconf
Since this parameter will be used on all daemon startups, it needs to beavailable on all nodes. This is achieved by querying it via ssconf. Thispatch additionally adds a getter method to readily retrieve the primaryip family from a ConfigWriter object....
Add new cluster parameter primary_ip_version
We expose the ip_version (4, 6) to the external interface and internallywe convert it to ip_family (AF_INET=2, AF_INET6=10). This makes the codemore concise as all functions deal with family rather than version....
netutils: make re class attribute and catch IndexError
These missing changes were initially agreed upon but then forgotten.First, we move the valid name regex to the class-level such that itwon't be compiled for every invocation of GetIP() and we wrap the result...
Conflicts: man/gnt-instance.sgml...
Fix some small newline style issues
RAPI client: Support modifying instances
RAPI: Allow modifying instance
Small fixes for instance creation via RAPI documentation
- Inconsistencies- Missing types
gnt-debug: Extend job queue tests
Test multiple opcodes, also with failure.
jqueue: Mark opcodes following failed ones as failed, too
When an opcode fails, the job queue would leave following opcodes as “queued”,which can be quite confusing. With this patch, they're all marked as failed andassertions are added to check this.
jqueue: Work around race condition between job processing and archival
This is a simplified version of a patch I sent earlier to make sure the jobfile is only written once with a finalized status.
rapi.client, http.client: Format url correctly when using IPv6
This patch moves the FormatAddress helper function from daemon.py tonetutils.py. This enables its use in http.client as well as inrapi.client. Furthermore this adds functionality to format IPv6...
Support IPv6 in lib/http/server.py
Support for resolving hostnames to IPv6 addresses
This patch enables IPv6 name resolution by using socket.getaddrinfoinstead of socket.gethostbyname_ex.
It renames the HostInfo class to Hostname and unifies its use throughoutthe code. This is achieved by using static calls where no object is...
Always use address instead of hostname in rpc.Client
In light of the upcoming IPv6 support, this patch enables the rpc.Clientto always use a node's address to connect to it. This is necessary as wedo not want to rely on name resolution to connect to the correct IP...
cluster init: Write ssconf before noded starts
This change is needed as we will need to read the primary ip versioncluster parameter before we start the node daemon. The reason is that weneed to know in advance if we bind to the IPv4 or IPv6 any address....
Introduce new IPAddress classes
This patch unifies the netutils functions dealing with IP addresses tothree classes:- IPAddress: Common IP address functionality- IPv4Address: IPv4 specific functionality- IPv6address: IPv6-specific functionality
Furthermore it adds methods to check whether an address is a loopback...
Add template 2.3 design doc
jqueue: More checks for cancelling queued job
We can also check when the lock status is updated. This willimprove job cancelling.
errors: Function to check whether value is encoded error
jqueue: Add more debug output
gnt-backup: Pass error code to OpPrereqError
Merge branch 'devel-2.1'
Re-add the 'live' parameter to migration opcodes
This patch reintroduces the live parameter, for backwards compatibilityat the Luxi level. This way, clients can work transparently with both2.1 and 2.2, even though sub-optimally.
Fix --master-netdev arg name in gnt-cluster(8)
This fixes Issue 114.
Restore 'tablet mouse on vnc' behavior
We needed to revert commit 5b062a58ac76b39c2dc6a7e1543affdf43dc7ee7because it was in conflict with the usb_mouse hv parameter. Here wereintroduce its functionality only when usb_mouse is not specified.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Document the usb_mouse hv parameter
Revert "Add -usbdevice tablet to KVM when using vnc"
This reverts commit 5b062a58ac76b39c2dc6a7e1543affdf43dc7ee7.This fixes issue 109. The mouse type can be set with the usb_mousehv parameter.
Fix path in ganeti-rapi man page
This takes care of issue 116.
workerpool: Don't keep reference to task arguments
The workerpool should not keep any reference to task arguments afterthey were processed by RunTask. Doing so led to jobs being cachedby the job queue's WeakValueDictionary for longer than they should've...
cli.SubmitOpCode: Pass keyword parameter as keyword
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Manuel Franceschini <livewire@google.com>
gnt-backup: Don't show confusing message w/o target node
“gnt-backup export” requires the target node. Until now, the masterdaemon would complain that the “parameter'OP_BACKUP_EXPORT.target_node' fails validation”. With this patch,an additional check is done in the client program....
masterd.instance: Add missing argument
_DiskTransferPrivate takes three parameters, not two.
Adjust message in case ~/.ssh is no directory
Use actual path, not something hardcoded.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
RAPI client: Fix docstring for migrating instance
QA: Test renaming instance via RAPI
RAPI client: Support renaming instances
Allow renaming instances via RAPI
RAPI client: Don't re-use PycURL object
With this patch, a new PycURL object will be created for each request.This should make the RAPI client safe for simultaneous calls frommultiple threads. Unittests are adjusted accordingly.
An unnecessary variable assignment is also removed from the unittest...
Add --no-name-check to 'gnt-instance rename' man page
Fix bug in bdev when drbd version format is x.x.x.x
This patch fixes a bug reported in [0]. Newer drbd versions can haveanother digit beyond the regular major, minor and point release digits.We modify the regex used to match that with an optional parts which is...
Bump version to 2.2.0~rc0
move-instance: Use constants for parameters
Allow instance NIC's IP address to be None
Also add some assertions.
Test instance NIC and disk parameter names
Add new parameter type “maybe string”
Before strict checking was implemented, NIC IP addresses could be setto “None”. Commit bd061c35 added more strict checking, includingenforcing the IP address to be a string. With this new type, itcan again be set to None....
cmdlib: Change expected type for source CA on remote import
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
move-instance: Pass OS parameters to new instance