InitConfig: create nodegroups as well
This patch also ensures that the initial configuration has all theneeded UUIDs and that they are unique, by using aTemporaryReservationManager inside InitConfit to generate them.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Export nodegroups list (names/uuids) via ssconf
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add nodegroup bash autocomplation
We autocomplete both by nodegroup name and uuid.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add nodegroup option to AddNode
Add node's nodegroup field
If a node doesn't have a node group we'll upgrade the config making itthe cluster default. Also the node add and removal operations arechanged to set/clear the node group correctly. Finally we populate the"members" list of nodegroups on config load with the value from the...
Check for nodegroup uuid indexing
Since the uuid is immutable the probability of it getting out of syncbetween the object and the dict key is very low. Still, checking ischeap, so we do it to be more sure nothing is wrong.
config.LookupNodeGroup
This function allows a node group to be looked up by name or uuid.If no nodegroup is specified and only one exists, that one is returned.
ConfigWriter: create the default node group
If no node groups exist we'll create a new default empty one.
Add a 'nodegroups' slot to ConfigData
Also: - reformat the slots declaration of ConfigData - call UpgradeConfig on each node groups to handle future upgrades - add nodegroups special case in {To,From}Dict - add nodegroups to _AllUUIDObjects
_ContainerFromDicts: handle None source
When _ContainerFromDicts is called on some element which doesn't existin the config, because it is yet to be upgraded, it will receive itsvalue as None. We take care of this case by using an empty element ofthe required target type....
Add a new NodeGroup config object
The "members" slot of this object is not serialized, and is discardedwhen deserializing, initializing it explicitly to [].
cli: Add option definition for priority
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
jqueue: Ensure only accepted priorities are allowed for submitting jobs
Quoting the design document: “Submitted opcodes can have one of the prioritieslisted below. Other priorities are reserved for internal use”. Submitting jobsat priority -20 should not be allowed....
Add support for job priority to opcodes and job queue objects
This allows clients to submit opcodes with a priority. Except for beingtracked by the job queue, it is not yet used by any code.
Unittests for jqueue._QueuedOpCode and jqueue._QueuedJob are provided for...
Add job priority constants
Merge branch 'devel-2.2'
Merge branch 'stable-2.2' into devel-2.2
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Fix pylint warning in http/__init__.py
My bad for not seeing this before:R0201:614:HttpBase.GetSslCiphers: Method could be a function
Allow SSL ciphers to be overridden in HTTP server
Users of this class, such as the RAPI server, might want to override or adjustthe default SSL cipher defined in a constant.
jqueue: Resume jobs from “waitlock” status
After an unclean restart of ganeti-masterd, jobs in the “waitlock” status canbe safely restarted. They hadn't modified the cluster yet.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
jqueue: Move queue inspection into separate function
This makes the init function a lot smaller while not changingfunctionality.
jqueue: Don't update file in MarkUnfinishedOps
This reduced the number of updates to the job files. It's used in two placeswhile processing a job and the file is updated just afterwards.
locking.SharedLock: Update class docstring
This was already outdated when the initial version of SharedLock was addedin commit 162c1c1f1 (February 2008).
locking: Implement priorities in SharedLock and LockSet
For proper support of job priorities, jobs' locks need to respectpriorities. Otherwise it could happen that a job with a lower prioritycould get a lock before a job with a higher priority (depending on...
Remove utils.EnsureDir as this is done by ensure-dirs.in now
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
The config now should also belong to confd group and readable by it
Move job queue to new ganeti.runtime
Revert "Make it possible to call utils.Daemonize with uid and gid to run as"
This reverts commit 743b53d4eb9f3de46edb5e54738dab287b1979ac.
Conflicts:
lib/daemon.py
Trivial conflict resolved. This patch reverts changes from earlier permissionsseparation stage. This is not needed anymore as start-stop-daemon takes care...
cli: Use list of options shared between commands
The completion script for bash has to know about these options. Until nowthe list was in two places--once in cli.py and once inautotools/build-bash-completion. A shared list is used with this patch.
jqueue: Use separate function for encoding errors
Comes with unittest.
Cluster.UpgradeConfig: populate primary_ip_family
Adding a runtime configuration library
This is used to expand the users/group names just once atinitial call.
Log warning instead of raising OpExecError for ndisc6
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix some epydoc warnings
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix breakage introduced by commit 8044bf655
Note to self: even patches removing one line can break everything.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Manuel Franceschini <livewire@google.com>
Remove “dry_run” from opcodes.OpCreateInstance
I'ts declared twice, once in opcodes.OpCode and here, and this oneis redundant.
Handle ENOENT case in ssconf.GetPrimaryIPFamily
This patch adds an optional default parameter to SimpleStore._ReadFile. Thiscan be used to default the return value of this method in case the ssconf fileis not present.
In this particular case it is used to return AF_INET in case...
Show list of pending acquires in “gnt-debug locks”
This is accomplished by keeping a list of waiting threads insteadof just their number inside the lock-internal condition. A fewother tweaks to the output format are also made.
Fix scp command when target is an IPv6 address
Due to the syntax used for the target in scp <target>:<path>, it isnecessary when the target is an IPv6 address to enclose it in squarebrackets.
Change bootstrap.SetupDaemonNode to use scp as we can assume SSH is setup
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
hansmi helped me with merging the conflict. Thanks
Conflicts: lib/workerpool.py
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Adding a paramiko fingerprint format helper
And provide unittests for them
workerpool: Add support for task priority
To add job priorities, the worker pool underlying the job queue mustsupport priorities per task. This patch adds them to the worker pool.
Add simple lock monitor
This patch adds an initial implementation of a lock monitor, accessiblefor the user through “gnt-debug locks”. It currently shows all resourcelocks: BGL, nodes and instances. Config and job queue locks could beshown too, but wouldn't be of much help. The current owner(s) and mode...
workerpool: Allow setting task name
With this patch, the task name is added to the thread name and will show up inlogs. Log messages from jobs will look like “pid=578/JobQueue14/Job13 mcpu:289DEBUG LU locks acquired/cluster/BGL/shared”.
Use one function to parse “--fields” option values
locking.LockSet: Use function to get member lock name
Switch to the RPC call to update /etc/hosts in LUAddNode and LURemoveNode
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add RPC calls to update /etc/hosts
Fix a few epydoc docstrings
Make family argument in FormatAddress optional
By doing this we delegate the task of finding the correct address familyto the FormatAddress method.
Support IPv6 for instances
gnt-node add: add error msg when using IPv6
Use family in backend.StartMaster
This patches changes the StartMaster method to consult the clusterprimary ip version when deciding whether to use arping or ndisc6 afteractivating the master ip.
Signed-off-by: Manuel Franceschini <livewire@google.com>...
Make Hostname object always intialize its name to fqdn
This patch restores the behaviour of Hostname (previously known asHostInfo) to always use fqdn. This was broken due to the fact that thenow used getaddrinfo does not return an fqdn in contrast togethostbyname_ex()....
Fix small spelling mistake
Stop adding the dry-run option by default
Currently cli.py unconditionally adds the dry-run option. This patchdisables this, and exports dry-run as a normal option.
The other alternative I tried to implement (adding a new fake option fordisabling the auto-add per individual command) would require changes in...
jqueue: Remove lock status field
With the job queue changes for Ganeti 2.2, watched and queried jobs areloaded directly from disk, rendering the in-memory “lock_status” fielduseless. Writing it to disk would be possible, but has a huge cost atruntime (when tested, processing 1'000 opcodes involved 4'000 additional...
utils: Use WriteFile in {Set,Remove}EtcHostsEntry
This avoids duplicate effort and has been a TODO for a long time.
Merge branch 'devel-2.1' into devel-2.2
Don't ignore secondary node silently
Currently on non-mirrored disk templates the secondary node is ignoredsilently. This patch adds a check for this case, and warns the usershould this be happening. This solves issue 113.
The patch also moves a prereq check to an argument check. This is ok...
Modify gnt-node add to call external script
Make it possible to invoke RunCmd in interactive mode
This is needed so we can run external scripts asking for password and suchto run interactive. Downside is that we can't get the output of the programbut we can still use the exit code.
Signed-off-by: René Nussbaumer <rn@google.com>...
Removing all ssh setup code from the core
Support IPv6 node add
Support IPv6 cluster init
Add primary_ip_family to ssconf
Since this parameter will be used on all daemon startups, it needs to beavailable on all nodes. This is achieved by querying it via ssconf. Thispatch additionally adds a getter method to readily retrieve the primaryip family from a ConfigWriter object....
Add new cluster parameter primary_ip_version
We expose the ip_version (4, 6) to the external interface and internallywe convert it to ip_family (AF_INET=2, AF_INET6=10). This makes the codemore concise as all functions deal with family rather than version....
netutils: make re class attribute and catch IndexError
These missing changes were initially agreed upon but then forgotten.First, we move the valid name regex to the class-level such that itwon't be compiled for every invocation of GetIP() and we wrap the result...
Conflicts: man/gnt-instance.sgml...
Fix some small newline style issues
RAPI client: Support modifying instances
RAPI: Allow modifying instance
gnt-debug: Extend job queue tests
Test multiple opcodes, also with failure.
jqueue: Mark opcodes following failed ones as failed, too
When an opcode fails, the job queue would leave following opcodes as “queued”,which can be quite confusing. With this patch, they're all marked as failed andassertions are added to check this.
jqueue: Work around race condition between job processing and archival
This is a simplified version of a patch I sent earlier to make sure the jobfile is only written once with a finalized status.
rapi.client, http.client: Format url correctly when using IPv6
This patch moves the FormatAddress helper function from daemon.py tonetutils.py. This enables its use in http.client as well as inrapi.client. Furthermore this adds functionality to format IPv6...
Support IPv6 in lib/http/server.py
Support for resolving hostnames to IPv6 addresses
This patch enables IPv6 name resolution by using socket.getaddrinfoinstead of socket.gethostbyname_ex.
It renames the HostInfo class to Hostname and unifies its use throughoutthe code. This is achieved by using static calls where no object is...
Always use address instead of hostname in rpc.Client
In light of the upcoming IPv6 support, this patch enables the rpc.Clientto always use a node's address to connect to it. This is necessary as wedo not want to rely on name resolution to connect to the correct IP...
cluster init: Write ssconf before noded starts
This change is needed as we will need to read the primary ip versioncluster parameter before we start the node daemon. The reason is that weneed to know in advance if we bind to the IPv4 or IPv6 any address....
Introduce new IPAddress classes
This patch unifies the netutils functions dealing with IP addresses tothree classes:- IPAddress: Common IP address functionality- IPv4Address: IPv4 specific functionality- IPv6address: IPv6-specific functionality
Furthermore it adds methods to check whether an address is a loopback...
jqueue: More checks for cancelling queued job
We can also check when the lock status is updated. This willimprove job cancelling.
errors: Function to check whether value is encoded error
jqueue: Add more debug output
Merge branch 'devel-2.1'
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Re-add the 'live' parameter to migration opcodes
This patch reintroduces the live parameter, for backwards compatibilityat the Luxi level. This way, clients can work transparently with both2.1 and 2.2, even though sub-optimally.
Signed-off-by: Iustin Pop <iustin@google.com>...
Restore 'tablet mouse on vnc' behavior
We needed to revert commit 5b062a58ac76b39c2dc6a7e1543affdf43dc7ee7because it was in conflict with the usb_mouse hv parameter. Here wereintroduce its functionality only when usb_mouse is not specified.
Revert "Add -usbdevice tablet to KVM when using vnc"
This reverts commit 5b062a58ac76b39c2dc6a7e1543affdf43dc7ee7.This fixes issue 109. The mouse type can be set with the usb_mousehv parameter.
workerpool: Don't keep reference to task arguments
The workerpool should not keep any reference to task arguments afterthey were processed by RunTask. Doing so led to jobs being cachedby the job queue's WeakValueDictionary for longer than they should've...
cli.SubmitOpCode: Pass keyword parameter as keyword
masterd.instance: Add missing argument
_DiskTransferPrivate takes three parameters, not two.
Adjust message in case ~/.ssh is no directory
Use actual path, not something hardcoded.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
RAPI client: Fix docstring for migrating instance
RAPI client: Support renaming instances