Limit the PVs used for disk growth with exclusive storage
The exclusive storage flag is passed along in the backend functions fordisk growth. LVM volumes handles the flag by reusing the same PVs, as disksare constrained by the number of their spindles....
Add exclusive_storage to blockdev_grow RPC
The flag is passed to the RPC, but it's still ignored by noded.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Index instances by their UUID
No longer index instances by their name but by their UUID in the clusterconfig. This change changes large parts of the code, as the followingadjustments were necessary: * Change the index key to UUID in the configuration and the...
Index nodes by their UUID
No longer index nodes by their name but by their UUID in the clusterconfig. This change changes large parts of the code, as the followingadjustments were necessary: * Change the index key to UUID in the configuration and the ConfigWriter, including all methods....
Add cluster name to instance migration RPC
This fixes a 'FIXME' in the xen hypervisor. The instancemigration requires the cluster's name. This should comefrom the RPC call, but so far was hackishly taken fromssconf. Ssconf is intended to be used by scripts only and...
Remove XEN_CMD from constants, adjust to PowercycleNode
This patch extend the RPC to powercycle a node by ahvparams parameter. This parameter is used by thexen hypervisor to determine which xen command to use.The respective LU is adjusted as well.
Since this was the last operation depending on the...
Use hvparams in GetAllInstancesInfo
This patch extends the GetAllInstancesInfo method of thehypervisors by a hvparams parameter. This change propagatesup to the RPC call all_instances_info, which also neededto be extended. All callers of this call were adjusted...
Use hvparams in GetInstanceInfo
This patch extends the GetInstanceInfo function of thehypervisors by an hvparams parameters. This leads toan extension of the instance_info RPC call and theadjustmend of all classes using this call.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Add hvparams to RPC call 'node_info'
This patch adds the hvparams parameter to the RPC call'node_info'. It also adjusts the related code in noded.pyand Query/Node.hs
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Verify: node info and instance list
This patch enhances node verification by:- Factoring out the verification of the node info and the instance list to increase testability.- Make both of them use the hvparams parameter.
Verify: xen toolstack, hypervisor and hvparams
This patch extends the node verification by:- Adding a check for the xen toolstack when the hypervisor is verified.- Factoring out the hypervisor verification in a subfunction to increase testability.- Factoring out the hvparams verification in a...
rpc / noded: add hvparams to instance_list call
The 'instance_list' call needs to be extended to accepthvparams. This is one of the few RPCs where it isnecessary, because there is no instance given whosehvparams could be reused.
Add generic daemon options support binding to interfaces
In some situations, it is desirable to bind one of our servers only toa particular interface, like eth0, instead of an IP address. Therefore,add an option -i to specify an interface; its address will then be bound...
New RPC to get size and spindles of disks
This RPC replaces the existing one that only returned disk size.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
RPC 'node_info': <storage_type,key> instead of vg_names
This replaces the field 'vg_names' in the RPC call of 'node info' by'storage_units'. A storage unit is a tuple <storage_type,key>and a generalization of a vg_name. The list of vg names is replaced by...
Move 'container.py' to storage directory
Moving 'container.py' to the storage directory.
Rename storage.py to container.py
Renaming 'storage.py' to 'container.py'. It will be moved into the new'storage' directory, which will come in later patches to avoid clashes ofnotation.
Reason trail implementation for "start"
Opcode-specific implementation of the reason trail for the instancestartup operation.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Reason trail implementation for "shutdown"
Opcode-specific implementation of the reason trail for the instance shutdownoperation.
Reason trail implementation for instance reboot
This commits allows ganeti to correctly forward the reason trail informationregarding instance reboot.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add function for extending the reason trail
Noded functions will have to extend the reason trail specifying whatoperation is being executed.
This commit adds a function for doing that (will be used by future commits).
Signed-off-by: Michele Tartara <mtartara@google.com>...
Remove old "reason" implementation
Remove the useless parts of the old, partial, implementation of the support fortracking the reason of instances state change, before implementing the newreason trail support, as per the design document.
Fix style violations in commit 4a90bd4, small changes
- Commit 4a90bd4 contained a rather large number of style violations: - Indentation/sequence formatting - Wrapping of long lines - Docstrings whose first line was wrapped - A stray backslash in a docstring...
RAPI: Add flag to require authentication
Most RAPI resources do not require authentication for the “GET” method.In some setups it can be desirable to always require authentication.This patch adds a command line parameter to always require it.
Some unrelated minor typos in the “ganeti-rapi” man page are also fixed....
Helper to retrieve access permissions for RAPI resource
Merge similar code into a helper function.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Status change reason support for Reboot
Add support to the Reboot command for specifying the reason for the laststatus change.
Some features are implemented as functions, even if used only once, becausethey will be used by the future patches introducing reason support for all...
New RPC call for node verify in dns-only RPC client
This will be used in node-add.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add the gnt-storage client
Add a new client called 'gnt-storage'.The client interacts with the ExtStorage interface, similarly tothe way gnt-os interacts with the OS interface.
For now, only two commands are supported: 'info' and 'diagnose'.
'diagnose' calculates the node status of each provider on each node,...
Add exclusive_storage parameter to node_info RPC
The flag is passed to the low-level functions that need it, but it's notyet used. Also, RPCs get just a default value for now.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add exclusive_storage parameter to blockdev_create RPC
The flag is passed to the low-level functions that need it, but it's notyet used.
Read watcher pause using RPC, not directly
The master daemon should not directly read files written by the nodedaemon. This patch adds a new RPC to read the watcher pause file andchanges the master code to use it.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Set watcher pause on all nodes
Instead of just setting the watcher pause file, which isn't replicated,RPC is used to set it on all nodes (where possible). This means thatafter an outage of the master node and a following master-failover, thewatcher will still be paused....
Add RPC for setting watcher pause
The watcher pause file should be set/unset on all nodes at once, notonly the master node. For that a new RPC is needed.
Replicate queue drain flag across all master candidates
Until now, the flag was unset on a master failover unless the“$localstatedir/lib/ganeti/queue/drain” file existed.
masterd: Remove duplicate code
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
backend: Rename RunRemoteCommand to RunRestrictedCmd
As per Iustin Pop's suggestion in <20121115131730.GX824@google.com> on<ganeti-devel@googlegroups.com>.
Implement LUNetworkQuery
Summarily list all existing networksSupply detailed info for every existing network - List used/free IPs - List instances with NICs assigned to the corresponding network - List NIC index and IP for the above instances
Implement complementary config methods for retrieving networks....
Basic IP pool management logic
Implement LUs for corresponding opcodes: * LUNetworkAdd: - Check for IP validity - Reserves all necessary IPs - Create new Network config object * LUNetworkRemove: - Checks if connected to any nodegroup - Remove a Network config object...
Add RPC for restricted commands
Expose the functionality to run restricted commands remotely via RPC.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Expose changing job priority via LUXI
A new LUXI request is added, in both Python and Haskell.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'devel-2.6'
Improve logging of new job submissions
This addresses issue 290: when receiving new jobs, logging isincomplete, and we don't have the job ID and/or summarieslogged. Only later, when the job is queried for or being processed, weknow more.
This is not good when troubleshooting, so let's improve the initial...
Update blockdev's "info" at instance rename
Currently, we set "info" metadata on block devices at device creationtime, but we never update it, leading to stale data in case ofinstance renames. This would not be a big problem in case of regularrenames (assuming this is a rare operation), but importing instances...
Remove support for PUT in noded
This takes care of a FIXME; 2.6 already uses the new method, so we'regood during upgrades.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Move a function from backend to ssconf
The “WriteSsconfFiles” function is used to write ssconf files. By movingit we can avoid importing backend into bootstrap. The latter is importedby CLI programs and backend doesn't have much to do with them.
Remove the python confd server side code
In 2.7 we will only support the Haskell version, if enabled.
Since the original hconfd enabling was a bit hack-ish (copying overthe actual installed ganeti-confd, Python version), the Makefile.amchanges are a bit more involved than just the removal of the Python...
Migrate lib/server/*.py from constants to pathutils
File system paths moved from constants to pathutils.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Bump pep8 version to 1.2
Debian Wheezy will ship with this version, and it has many improved checks compared to 0.6, so let's:
- bump version in the docs- silence some new checks that are wrong due to our indent=2 instead of 4- fix lots of errors in the code where the indentation was wrong by 1...
Fix uses of OpPrereqError without code info
A while back, we did cleanup the code and ensured (manually) that useof OpPrereqError includes an errors.ECODE_* field as secondargument. Since we cannot automate the check for this, it turns outthat more and more such usage has crept over the years, including in...
Merge branch 'stable-2.6'
Another small consistency fix with if branches
While looking at the testability of this piece of code, I found anotherconsistency problem. We have two if branches instead of one, withelif's.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix inconsistency in the LUXI protocol w.r.t. args
This inconsistency was found during rebalancing. Hbal failed because,Ganeti couldn't load the opcode. After digging through the cause, aninconsistency with the "args" field in the LUXI protocol was triggered...
Fix DRBD resize code
There are two bugs in the current resize code, affecting mostly DRBD.
First, due to bugs in old DRBD versions (pre 8.0.14), the code currentlycalls `drbdsetup resize' on both the primary or secondary. However,this is actually wrong per current DRBD (from drbdsetup(8)):...
Correct capitalisation of two Luxi calls
Two Luxi calls have inconsistent an name/value mapping (in the Pythoncode):
- REQ_AUTOARCHIVE_JOBS versus AutoArchiveJobs (versus AutoarchiveJobs)- REQ_QUEUE_SET_DRAIN_FLAG versus SetDrainFlag (no Queue)
While these are only a consistency issue, let's fix them so that the...
errors: Add exception for RAPI testing utilities
This exception is raised to abort before actually sending a LUXI call(there is no LUXI server involved in the test). The testing utilitiescatch the exception to report a success (i.e. the code didn't throw...
server.rapi: Factorize RAPI user loading
By moving the code loading RAPI users from a file into a standalonefunction it becomes easier to use the RAPI request handler intests.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
RemoteApiHandler: Add support for mocked LUXI client
This will be used for providing a type-checking RAPI client for tests.
Stop acquiring BGL for LUXI queries
Short description: This fixes an issue whereby masterd would becomeunresponsive on the LUXI socket, leading to client timeouts. While madeworse in 2.5, the underlying issue was already present in 2.4.
Longer description: Until now all LUXI queries would acquire the BGL...
Stop using locks in LUXI "QueryTags"
Also mark it as deprecated in NEWS as normal queries can be usedinstead.
Drop objects.QueryRequest
It was only used in one place and wasn't really necessary.
Merge branch 'devel-2.5'
Add job support to query2 via LUXI
This enables the use of filters through query2 when listing jobs.
cmdlib: Stop forking in LUClusterQuery
While debugging another issue we realized that LUClusterQuery forks.This turned out to be the “platform.architecture” function from thePython library. It uses the “file” command to determine the architectureof the Python binary....
http.server: Move error message formatting to handler class
Like before this patch moves more functionality from the actual serverclass into a separate handler class. At the same time the function ischanged to return both content-type and body instead of relying on a...
noded: Shorter docstring for mlockall(2) executor
Split handling HTTP requests into separate class
Until now HTTP requests were handled in the same class as incomingconnections (http.server.HttpServer). With this change the requesthandling is delegated to a separate class which can be re-used in tests...
Remove deprecated “QueryLocks” LUXI request
This has been deprecated since Ganeti 2.4 and hasn't been used byGaneti's code since. I'm not aware of any external users.
Accept both PUT and POST in noded
This is a partial cherry-pick from7530364ddbe949bc34fc26f25ba3f5d921beb021 on master:
Currently, noded requires PUT, even though the semantics of the RPCcalls do not match a PUT. We change the code accept both PUT and POST,...
Add instance_balloon_memory rpc
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Currently, noded requires PUT, even though the semantics of the RPCcalls do not match a PUT. We change the code accept both PUT and POST,with the intention to remove the PUT support in a later version.
On the client side, the RPC code is changed to send POST requests....
rpc: Change signature of RpcRunner
Instead of receiving the whole masterd context, it only gets theconfiguration and the lock monitor callback. This simplifiesunittesting.
Additionaly, two new arguments for testing are added.
Switch blockdev_getmirrorstatus_multi to per-node bodies
Even for a small 4-node node-group, this reduces the payload of thisRPC call by half for each node; for a big node group, previous testing(using a different patch) showed a general reduction in traffic over...
serializer: Remove JSON indentation and dict key sorting
Serializing to JSON using “simplejson” is significantly slower whenindentation and/or sorting of dictionary keys is used. In simplejson 1.xthe difference isn't that big, but with simplejson 2.x the difference...
Change “node_info” RPC to accept multiple VGs/hypervisors
Keeping the node state up to date will require information from multipleVGs and hypervisors. Instead of requiring multiple calls this changeallows a single call to return all needed information. Existing users...
masterd: Initialize job queue only after RPC client
Otherwise jobs started after an unclean master shutdown will fail asthey depend on the RPC client.
masterd: Shutdown only once running jobs have been processed
Until now, if masterd received a fatal signal, it would start shuttingdown immediately. In the meantime it would hang while jobs are stillprocessed. Clients couldn't connect anymore to retrieve a jobs' status....
masterd: Don't pass mainloop to server class
It is not used.
Change master IP address RPCs for external script
Change the master IP address RPC call chain to accept theuse_external_master_ip_script parameter. Introduces an unused parameterin backend.ActivateMasterIp and backend.DeactivateMasterIp, that will beused in the next commit....
Show RPC calls from config in lock monitor
With this patch all RPC calls at runtime of masterd will show up in thelock monitor. There is a chicken-and-egg issue with initializing theconfiguration with a context since the lock manager, containing themonitor, requires the configuration. This is worked around by setting...
Pass MasterNetworkParameters instances in RPCs
Pass instances of objects.MasterNetworkParameters when calling RPCs foractivation and deactivation of master IP.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Use MasterNetworkParameters attributes for RPC
Instead of manually unpacking the return values ofcfg.GetMasterNetworkParameters, let it return an instance ofobjects.MasterNetworkParameters and pass its attributes.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>...
Uniform master IP activation and deactivation
Add the master IP family parameter to the master IP deactivation RPCs,so that the activation and deactivation interfaces are uniform.
Explicitly pass params to change_master_netmask
Make the master explicitly pass the parameters to thechange_master_netmask RPC, and change all the call flow to use the newinterface.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Explicitly pass params to deactivate_master_ip
Make the master explicitly pass the parameters to thedeactivate_master_ip RPC, and change all the call flow to use the newinterface.
Change the semantics of stop_master RPC
To avoid polluting the stop_master RPC interface with networkparameters, this patch removes the deactivation of the master IP fromthis RPC.
The call to deactivate_master_ip is then added before every stop_master...
Explicitly pass params to activate_master_ip
To remove the usage of ssconf in the backend, the master needs to pushthe parameters of activate_master_ip to the backend.
This patch changes the entire call path of activate_master_ip to use thenew interface....
Add new lock level for node resource modifications
This is in preparation for implementing a new resource model.
rpc: Change {import,export}_start to take source/dest in single argument
This simplifies the RPC argument encoding code.
rpc: Convert last two calls to generated code
These two calls, “upload_file” and “write_ssconf_files” are treatedseparated as they're used by the configuration, where we can't use thenormal resolver.
There's still some duplicated code in rpc.py, but that will be taken...
Move bootstrap-related RPC to generated wrappers
With this patch, only 6 RPC are left as old-style code.
Convert job queue's RPC to generated code
With these changes job queue RPC will finally show up on the lockmonitor. See below for an example. A job queue-specific class is used torestrict the use of a static list for name resolution to the job queue.Further improvements can be made to not re-create the whole RPC client...
rpc: Convert blockdev-related calls
Fix parameters to RPC "blockdev_rename"
Before this patch it would take a variable number of arguments, whichisn't very good for a definition-based approach.
Remove “node_tcp_ping” RPC
It is not used anywhere.
Standardise LUXI call argument types
Currently, we have 4 types of arguments in LUXI calls:
- most common, a list of values- a single argument that is sent as a list of one element- a single argument that is sent by itself- a dictionary (only Query and QueryFields)...
Rename filter and filter_ to qfilter
We currently use 'filter' as the OpCode, QueryRequest and RAPI fieldname for representing a query filter. However, since 'filter' is abuilt-in function, we actually have to use filter_ throughout the codein order to not override the built-in function....
Add cluster netmask parameter
Add the master_netmask cluster parameter, that represents the netmask ofthe master IP, encoded as a CIDR suffix.
This parameter can be set via the --master-netmask of gnt-cluster initand gnt-cluster modify. The default behaviour is to be consistent with...
Split starting and stopping master IP and daemons
Make migration RPC non-blocking
To add status reporting for the KVM migration, the instance_migrate RPCmust be non-blocking. Moreover, there must be a way to represent themigration status and a way to fetch it.