Add tags in network objects
Support: - gnt-network add --tags. - gnt-network list-tags/add-tags/remove-tags/. - gnt-network list -o +tags
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Iustin Pop <iustin@google.com>
Change default fields for network listing
Pass netinfo in rpcs
If a nic has a network field then encapsulate a network object innetinfo slot for every rpc. This is needed to pass network info toscripts managing nics (kvm-vif-bridge).
Introduce _BuildNetworkEnv().
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>...
Pass detailed network info in hooks
Use network's mac prefix
Modify existing GenerateMAC so that it takes network as an argument.
Substitute _GenerateOneMAC with a decorator _GenerateMACPrefix thatchooses the prefix (network's or cluster's) and wraps_GenerateMACSuffix that generates the three remaining bytes....
Rapi support for networks
Support: - GetNetwork(s) - CreateNetwork - ConnectNetwork - DisconnectNetwork - RemoveNetwork
ss_conf support for networks
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Iustin Pop <iustin@google.com>
Introduce client support for networks
gnt-network is used to manipulate and handle networks that currentlyprovides the following operations:
Modify instance client to support networks
Add --no-conflicts-check option.Modify instance query methods to return nic's network info.
Peripheral changes for networks
Specifically: * export a new environment variable for the gnt-os-interface * export a new environment variable for KVM specific scripts * ovf support for networks
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>...
Modify LUInstanceSetParams() to support networks
Keep backwards compatibility just like LUInstanceCreate() and supportnetwork parameter for NIC object.
Add ec_id argument in Update() to be able to commit temporary IPs thatwere previously reserved.
Modify RemoveInstance() to support networks
Release any IPs held the instance before actually removing.
Modify LUInstanceCreate to support networks
Implement backend support, to export the IP pool managementfunctionality to the clients. When the new NIC parameter 'network' isgiven, the ippool management system is triggered. If a NIC belongs toa network, it inherits the netparams (mode, link) as its nicparams. If...
Config methods for reserving/releasing IPs
Use TemporaryReservationManager for IP handling to ensure config dataconsistency.
Implement: - methods for reserving/releasing IPs - _UnlockedCommitTemporaryIps() - CheckIPInNodeGroup() used for conflicting IPs...
Implement network/nodegroup mappings
LUNetworkConnect() connects a network to a nodegroup and definesits netparams (mode and link). Specifically: * Check if network already exists * Check if netparams are valid * Check if already connected to the same nodegroup...
Implement LUNetworkSetParams
Support modifying all network parameters except for network (iprange). Cannot modify gateway and reserved ips at the same time.
Implement LUNetworkQuery
Summarily list all existing networksSupply detailed info for every existing network - List used/free IPs - List instances with NICs assigned to the corresponding network - List NIC index and IP for the above instances
Implement complementary config methods for retrieving networks....
Basic IP pool management logic
Implement LUs for corresponding opcodes: * LUNetworkAdd: - Check for IP validity - Reserves all necessary IPs - Create new Network config object * LUNetworkRemove: - Checks if connected to any nodegroup - Remove a Network config object...
Introduce new module for IP pool management
Add new library module lib/network.py.Introduce new class: AddressPool.
AddressPool implements all operations needed for managing IPs insidethe IP pool.
Given a Network config object (nobj), the class:
IP pool related objects, opcodes and constants
Config objects: * Introduce L{Network} with - IPv4 network field (mandatory) - IPv4 gateway, IPv6 (network/gateway), mac prefix, type (optional) * Modify existing config objects to support networks: - Add new slot 'network' to L{NIC} config object...
Improve error message for wrong opcode parameter values
When given an empty string as value to a parameter that doesn't acceptempty strings, the error message was confusing, since it was showingjust as a double space.
Signed-off-by: Iustin Pop <iustin@google.com>...
Cleanup ht's use of positive/strictpositive
Currently, ht.py uses a bad terminology for positive/non-negativenumbers. Per http://en.wikipedia.org/wiki/Positive_number, this is thecorrect terminology:
- A number is positive if it is greater than zero.- A number is negative if it is less than zero....
locking: Simplify condition
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add opcode for running commands remotely
The opcode doesn't pay attention to the build-time flag to enable ordisable restricted commands. In a cluster different nodes could havedifferent settings.
Node locks are acquired in shared mode by default, but the use of an...
Add RPC for restricted commands
Expose the functionality to run restricted commands remotely via RPC.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
locking: Don't schedule pending acq. for short timeout
Scheduling a pending acquisition is relatively expensive and lot of codeis involved. Unless there is already one, a new pipe needs to be opened.Data structures need to be updated as well, only to be undone shortly...
backend: Implement remote commands
As per design document (doc/design-remote-commands.rst), a number ofrather strict tests is applied to any incoming request, a delay isinserted upon errors and returned error messages are very generic(unless it's the actual command that failed). There are unit tests for...
configure: Add option to enable remote commands
By default remote commands are disabled and need to be explicitelyenabled at build time.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
rapi: Add new user option for querying
This was requested in issue 301. Before this patch, requests to“/2/query/*” and “/2/instances/*/console” would require authenticationwith a user with write access. Since that is not strictly necessary, anew user option named “read” is added....
Move gnt_cluster.SHOW_MACHINE_OPT to cli
This allows the option to be re-used in other places.
Warn on invalid lines in HTTP user files
Without this change, invalid lines or values would be silently ignored.
pathutils: Add directory for remote commands
Also add tests to ensure it's never allowed as a file storage path. Aconstant for the lock file is also added.
utils.process.RunResult: Always set "fail_reason" attribute
Expose changing job priority via LUXI
A new LUXI request is added, in both Python and Haskell.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add “gnt-job change-priority” sub-command
This can be used to change the priority of a pending or running job (thelatter only if there are unprocessed opcodes).
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
workerpool: Add method to change task's priority
Using the task ID a pending task's priority can be changed. This will beused to change the priority of jobs in the workerpool.
workerpool: Preserve task number when deferring
When a task is deferred it should receive the same task ID upon beingreturned to the pool.
jqueue: Set task ID for jobs added to workerpool
The job ID is re-used as the task ID, as job IDs are unique.
jqueue: Allow changing of job priority
This is due to a feature request. Sometimes one wants to change thepriority of a job after it has been submitted, e.g. after submitting animportant job only to later notice many other pending jobs which will beprocessed first. Priority changes only take effect at the next lock...
workerpool: Change data structure for priority change
To prepare for the addition of a new function allowing changing apending task's priority, the internal data structure is slightlychanged. The (optional) task ID is stored as part of the task entry. A...
Documentation for the NODE_RES level
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
RunCmd: Expose "postfork" callback
The “_postfork_fn” parameter was only used for tests until now. Toimplement a good locking scheme, remote commands must also make use ofthis callback to release a lock when the command was successfullystarted (but did not yet finish)....
Merge branch 'devel-2.6' into master
Conflicts (both trivial): htools/Ganeti/Daemon.hs (_writePidFile rename)...
Merge branch 'stable-2.6' into devel-2.6
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Improve error message when migration status fail
Commit 6a1434d7 (“Make migration RPC non-blocking”) changed the APIfor reporting migration status, but has a small cosmetic bug: if themigration status if failure, but the RPC itself to get the statusdidn't fail, it shows the following error message:...
Fix type error in kvm/GetMigrationStatus
Commit 6a1434d7 (“Make migration RPC non-blocking”) changed fromraising HypervisorErrors to returning MigrationStatusobjects. However, these objects don't have an "info" attribute, sothey can't pass a reason back (which is in itself a bug); but the KVM...
sphinx_ext: Allow use of “rapi” module in pyeval
This way constants like “rapi.RAPI_ACCESS_WRITE” can be used indocumentation.
rlib2: Document two previously undocumented functions
Commit 208a6cff just included empty docstrings.
jqueue/mcpu: Determine priority using callback
Instead of being given the priority for acquiring locks by means of aparameter, mcpu will now call back. This is in preparation forimplementing a command to change a job's priority on the fly and allowsto change it while locks are being acquired (taking effect on the next...
Merge branch 'devel-2.6'
http/__init__.py: Remove extraneous argument
pylint complained, I fixed it, and unfortunately pushed too early.
rapi.testutils: Add utility to format HTTP headers
Once again this will be used by forthcoming RAPI test.
rapi.testutils: Return headers from mock utility
A newly added test for RAPI will also verify the returned headers. Atest in ganeti.rapi.client_unittest.py is split into smaller stand-alonetests.
http: Add wrapper for mimetools.Message
A newly added piece of code will also have to parse headers, so havingthis wrapper saves us from copying this part of code.
Add missing tests for commit f0d2286
Commit f0d2286 changed the logic ofgnt_instance._ConvertNicDiskModifications to also allow a parameternamed “modify”. Unfortunately the corresponding unittest was notupdated. An “if”/“else” condition is also merged....
workerpool: Use itertools.count instead of manual counting
Instead of having to explicitely increment the value (“… += 1”), a callto next() is enough. These numbers should in no case be re-used (theyare used for ordering tasks). Using “itertools.count” is useful here as...
Merge branch 'devel-2.6' into submit
Conflicts: Makefile.am (reordering, fixed) htools/Ganeti/Confd/Server.hs (hlint fixes on master) htools/Ganeti/Daemon.hs (hlint)...
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Use SSH_LOGIN_USER rather than root for xl ssh
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix gnt-instance console with xl
- Rename xm-console-wrapper to xen-console-wrapper- Pass the xen command to use as a parameter
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add utility to check if file is executable
This replaces direct calls to “os.access” and“os.path.exists”/“os.path.isfile”.
Fix NameError in constants.py introduced in merge 46c1f82
Conflicts: lib/hypervisor/hv_xen.py: trivial
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>...
Disable E1101 on ganeti/http/server.py:424
Fix live migration under xl
Until now the only way to make live migration work in conjunction with"xl" was to add ssh known_hosts keys for every node's secondary ip onevery other node.
With this command we remove the target key verification: this is not...
Don't check for xend port when using xl
If the toolstack is set to "xl" we shouldn't ping xend for livenessbefore attempting a live migration.
utils.io: Improve handling of double and single slashes
Up until now “IsBelowDir("/", …)” would never return True. The reasonwas that an additional slash was added to the root path resulting in“//", which is “implementation-defined” in posix and treated specially...
workerpool: Don't mask variable in AddManyTasks
The name “priority” is already used.
workerpool: Simplify _WaitForTaskUnlocked
The function in is simplified in its structure and duplicated checkshave been merged.
cli.py: use None as name for tag operations on the cluster
This change is mostly cosmetic. Previously, the literal "cluster" wasused for the 'name' field of tag operations on the cluster (as opposedto a node or an instance). Since this field has a type of TMaybeString...
Fix previous merge
A call to _CalculateGroupIPolicy wasn't refactored during the merge.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
jqueue: Return jobs to queue when shutting down
When a job is still waiting for locks and the queue is shutting down,they should be returned and not actually start processing. Until nowjobs which transitioned from “queued” to “waiting” were alreadyconsidered to be running as far as the shutdown code was concerned....
gnt-debug delay: Add "--submit" option
Make hostname checks uniform between instance rename and add
Currently, we have instance rename doing extra checks on the hostname, to prevent accidental wrong renames; however, instance createdoesn't do these checks (issue 291), which (if DNS is misconfigured)...
Improve logging of new job submissions
This addresses issue 290: when receiving new jobs, logging isincomplete, and we don't have the job ID and/or summarieslogged. Only later, when the job is queried for or being processed, weknow more.
This is not good when troubleshooting, so let's improve the initial...
Improve handling of lock exceptions
There are two issues with lock exceptions right now:
- first, we don't log the original error; this is fine for now (locking.py always returns the same error here), but in general is brittle: if locking.py would start returning more information, we'd...
Fix runtime memory increases
Commit 2c0af7da which added the runtime memory changes functionalityhad a small typo (wrong name); I've rewritten this to only compute thedelta once, for simplicity.
Fix validation of vgname in OpClusterSetParams
This variable can be empty, when we want to disable LVM, so we can'tuse TMaybeString.
Fixes issue 285.
Fix removal of storage directory on shared file storage
This patch makes _RemoveDisks symmetric to _CreateDisks with respectto file-based storage: _CreateDisks uses "in constants.DTS_FILEBASED",whereas _RemoveDisks was not update and only uses "==constants.DT_FILE". This results in stale directories left on the...
Switch non-redundant check to disk template-based
Currently, the warning/notice about non-redundant instances in clusterverify is based non empty secondaries list (how old is this?); theproper way to check this nowadays is via DTS_MIRRORED.
Fix permission for socket directory
The directory must we writable also by the confd daemon user.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add option to force master-failover without voting
This fixes issue 282.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
backend: Switch to new file storage directory verification
The configuration is no longer used for verifying file storage paths.
Check allowed file storage paths during cluster-verify
Some paths, such as /bin or /usr/lib, should not be used for filestorage. This patch implements a check during cluster verification tofail in case such a path has been used.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Make Paramiko an optional dependency for listrunner
With the move away from “setup-ssh”, Paramiko is no longer necessary toconfigure SSH on nodes.
Remove setup-ssh
It has been superseeded by “prepare-node-join”.
gnt-node add: Use prepare-node-join
This patch changes “gnt-node add” to use the newly added“prepare-node-join” tool. Hereby Paramiko is no longer a hard dependencyfor setting up SSH on nodes.
In “gnt_cluster.py”, a positional parameter is no longer passed as a...
prepare-node-join: Use ssh.GetAllUserFiles
Instead of building the dictionary locally, the global version in“ssh.py” can be used.
ssh: Add function to get all of user's SSH files
This new function returns the file paths for all of a user's SSH-relatedfiles (RSA, DSA and authorized_keys).
RunCmd: Support standard input file descriptor
This patch changes “utils.RunCmd” to accept a file-like object or anumeric file descriptor which will be used as the command's standardinput. One use-case will be to pass all necessary data to“prepare-node-join”....
Factorize job selection in “gnt-job cancel”
This will also be used for changing jobs' priorities. All parameters tothe common function are non-optional.
utils.x509: Factorize code to extract X509 certificate
This will be useful in “gnt-node add”.
prepare_node_join: Move daemon SSH files to constants
This dictionary will also be useful in “gnt-node add”.
prepare-node-join: Swap private and public keys
Other places, such as “ssh.GetUserFiles”, use a structure where theprivate key comes before the private key. Until now prepare-node-joindid the opposite, that is the public key came first. To avoid confusion...
prepare-node-join: Use public key directly for auth…_keys
A public key already includes the necessary prefix (“ssh-rsa” or“ssh-dss”), so there is no need to add it again.
ssh.GetUserFiles: Parameter to disable directory check
Without this parameter, either an error would be raised or “.ssh” wouldhave to be created. Now it is possible to retrieve the paths withoutrequiring the “.ssh” directory to exist.