Remove explicit reference to the query socket
Now that luxid's socket is the default socket anyway, do notpass the "query=True" parameter to GetClient. This will allowto get rid of this keyword argument, as masterd will go away.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
luxid: give stern warnings about debug mode
Luxid as it is can leak private and secret parameters by loggingall requests as they arrive, before any preprocessing is done.
Warn the user stern warnings about this.
Signed-off-by: Santi Raffa <rsanti@google.com>...
Simplify 'GetMasterInfo' RPC
RPC 'GetMasterInfo' returns several fields, namely, 'master_netdev','master_ip', 'master_netmask', 'master_node', and 'primary_ip_family',of which only the 'master_node' is actually used.
Correct exception when ssconf file does not exist
After an upgrade to 2.11, the ssconf file for the mastercertificates might not exist. Based on the non-existance,noded falls back to a compatibility mode regarding dealingwith SSL certificates. The check for the ssconf file...
Verify incoming RPCs against candidate map
From this patch on, incoming RPC calls are checked againstthe map of valid master candidate certificates. If no mapis present, the cluster is assumed to be inbootstrap/upgrade mode and compares the incoming call...
Extend RPC call to create SSL certificates
So far the RPC call 'node_crypto_tokens' did only retrievethe certificate digest of an existing certificate. Thiscall is now enhanced to also create a new certificate andreturn the respective digest. This will be used in various...
Retrieve a node's certificate digest
In various cluster operations, the master node needs toretrieve the digest of a node's SSL certificate. For thispurpose, we add an RPC call to retrieve the digest. Thefunction is designed in a general way to make it possible...
Replace errors re-export in luxi.py with proper imports
Instead of re-exporting errors in luxi.py, import rpc/errors.py in themodules that use them.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Split Luxi Client into a generic and a specific part
The generic part will be reused in WConfd.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add a Python directory for RPC code to keep it at one place
Move rpc.py to rpc/node.py and modify imports in existing code.
Add default_iallocator_params cluster parameter
Add a cluster parameter to hold the iallocator parameters usedby the default instance allocator. Implement the option tomodify config.data, query config.data and upgrade man pages,tests and cfgupgrade tool. The new default_iallocator_params is...
Instance queries: remove opcodes and LU
Removes the remains of the instance queries.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Remove op queries from masterd
Masterd still had some 'if' branches for the queries thatused to be op queries, but are now luxi queries. Thispatch removes them.
masterd: implement query via luxi
The master daemon so far still did queries via the pythonimplementation. This patch implements that it uses thehaskell implementation and removes the node queries fromthe list of OP-queriable entities.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Use custom SSH ports in node groups when working with nodes
Calling `gnt-instance console` with a custom SSH port doesn't work yet.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Add Luxi Request to pick up a job in the queue
During the transition to the new daemon layout, from step 2onwards, luxid will write to the queue but masterd will triggerthe execution. Therefore, add a new luxi request to tell masterdto pick up a job that has already been written to the queue....
Merge branch 'stable-2.10' into master
Check if hotplug is supported in CheckPrereq
Introduce new RPC hotplug_supported that invokes the correspondinghypervisor's method which checks if hotplug is generallysupported. Call this RPC early in CheckPrereq() and abort ifhotplug is not supported. Currently only KVM hypervisor with...
Remove blockdev_export RPC
This RPC call is no longer used, as for intra-cluster instance moves theimport/export daemon is used too.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Hotplug: rpc support
Introduce new RPC that eventually invoke hypervisor specifichotplug functions. In order to be generic it has the followingarguments: device type, action, device, extra info, seq.Device type can be NIC or DISK, action can be ADD, REMOVE,...
Add a console information RPC call
As the instance queries need console information, and the informationis retrieved through python classes that should not be ported yet, anRPC call supplying the information has been added. Some tests as well.
Signed-off-by: Hrvoje Ribicic <riba@google.com>...
Provide means of submitting jobs to a drained queue
During an upgrade, the job queue needs to be drained in order to avoidnew jobs coming to the cluster. Nevertheless, the upgrade processneeds to carry out some maintenance, like redistributing the new...
Use secondary IP when moving instances
All data traffic usually goes over the secondary network, butgnt-instance move didn't. This patch corrects this problem by using thetarget nodes secondary IP as move target.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Move 'BuildVersion' to 'lib/utils/version.py'
Functions 'BuildVersion' and 'SplitVersion' are no longer needed bythe constants and, given that they are not constants, they should bemoved elsewhere. Since they are only used by 'cfgupgrade' and tests,these functions are moved to 'lib/utils/version.py' and references to...
Replace physical_id with dynamic_params
The disk field 'physical_id' has to be kept up to date whenever a diskobject is sent to a node via RPC. This is done with the SetDiskID methodmanually, which is a source of bugs.
This patch replaces the use of 'physical_id' with a new field names...
Merge branch 'stable-2.9' into master
Add RPC functionality for creating OpenvSwitches
This patches adds the needed RPC functions in order to create theOpenvSwitches on the nodes.
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Fix bug in NodeD and RapiD usage strings
The strings did not comply with the style guide, and were not diplayedcorrectly either.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Typo fix in lib/server/noded.py
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>Reviewed-by: Guido Trotter <ultrotter@google.com>
Merge branch 'stable-2.8' into master
Change method dispatch in ClientOps to enforce luxi.REQ_ALL
ClientOps' handle_request dispatches on the luxi request received. Changethis to first verify if the request is luxi.REQ_ALL. In this way, we catchprogramming errors introducing "secret" luxi requests earlier. This is relevant,...
Check DRBD status on verify-disks
Enhance `gnt-cluster verify-disks` such that it checks the DRBD statusof all disks. If the status of at least one disk of an instance iseither StandAlone or Diskless, the instance disks are activatedautomatically.
noded.py: remove fallback of 'node_info'
In order to change backend and frontend code of the RPCcall 'node_info' in separate patches, we included afallback to the old implementation. Since all RPC callersare adjusted, this can be removed now.
Add general storage parameters to node info call
As described in the design doc about storage types,we plan to generalize the RPC call "node info" wrt tostorage types. This patch extends the call by acceptinga list of storage units, that is not only identified...
Limit the PVs used for disk growth with exclusive storage
The exclusive storage flag is passed along in the backend functions fordisk growth. LVM volumes handles the flag by reusing the same PVs, as disksare constrained by the number of their spindles....
Add exclusive_storage to blockdev_grow RPC
The flag is passed to the RPC, but it's still ignored by noded.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Index instances by their UUID
No longer index instances by their name but by their UUID in the clusterconfig. This change changes large parts of the code, as the followingadjustments were necessary: * Change the index key to UUID in the configuration and the...
Index nodes by their UUID
No longer index nodes by their name but by their UUID in the clusterconfig. This change changes large parts of the code, as the followingadjustments were necessary: * Change the index key to UUID in the configuration and the ConfigWriter, including all methods....
Add cluster name to instance migration RPC
This fixes a 'FIXME' in the xen hypervisor. The instancemigration requires the cluster's name. This should comefrom the RPC call, but so far was hackishly taken fromssconf. Ssconf is intended to be used by scripts only and...
Remove XEN_CMD from constants, adjust to PowercycleNode
This patch extend the RPC to powercycle a node by ahvparams parameter. This parameter is used by thexen hypervisor to determine which xen command to use.The respective LU is adjusted as well.
Since this was the last operation depending on the...
Use hvparams in GetAllInstancesInfo
This patch extends the GetAllInstancesInfo method of thehypervisors by a hvparams parameter. This change propagatesup to the RPC call all_instances_info, which also neededto be extended. All callers of this call were adjusted...
Use hvparams in GetInstanceInfo
This patch extends the GetInstanceInfo function of thehypervisors by an hvparams parameters. This leads toan extension of the instance_info RPC call and theadjustmend of all classes using this call.
Add hvparams to RPC call 'node_info'
This patch adds the hvparams parameter to the RPC call'node_info'. It also adjusts the related code in noded.pyand Query/Node.hs
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Verify: node info and instance list
This patch enhances node verification by:- Factoring out the verification of the node info and the instance list to increase testability.- Make both of them use the hvparams parameter.
Verify: xen toolstack, hypervisor and hvparams
This patch extends the node verification by:- Adding a check for the xen toolstack when the hypervisor is verified.- Factoring out the hypervisor verification in a subfunction to increase testability.- Factoring out the hvparams verification in a...
rpc / noded: add hvparams to instance_list call
The 'instance_list' call needs to be extended to accepthvparams. This is one of the few RPCs where it isnecessary, because there is no instance given whosehvparams could be reused.
Add generic daemon options support binding to interfaces
In some situations, it is desirable to bind one of our servers only toa particular interface, like eth0, instead of an IP address. Therefore,add an option -i to specify an interface; its address will then be bound...
New RPC to get size and spindles of disks
This RPC replaces the existing one that only returned disk size.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
RPC 'node_info': <storage_type,key> instead of vg_names
This replaces the field 'vg_names' in the RPC call of 'node info' by'storage_units'. A storage unit is a tuple <storage_type,key>and a generalization of a vg_name. The list of vg names is replaced by...
Move 'container.py' to storage directory
Moving 'container.py' to the storage directory.
Rename storage.py to container.py
Renaming 'storage.py' to 'container.py'. It will be moved into the new'storage' directory, which will come in later patches to avoid clashes ofnotation.
Reason trail implementation for "start"
Opcode-specific implementation of the reason trail for the instancestartup operation.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Reason trail implementation for "shutdown"
Opcode-specific implementation of the reason trail for the instance shutdownoperation.
Reason trail implementation for instance reboot
This commits allows ganeti to correctly forward the reason trail informationregarding instance reboot.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add function for extending the reason trail
Noded functions will have to extend the reason trail specifying whatoperation is being executed.
This commit adds a function for doing that (will be used by future commits).
Signed-off-by: Michele Tartara <mtartara@google.com>...
Remove old "reason" implementation
Remove the useless parts of the old, partial, implementation of the support fortracking the reason of instances state change, before implementing the newreason trail support, as per the design document.
Fix style violations in commit 4a90bd4, small changes
- Commit 4a90bd4 contained a rather large number of style violations: - Indentation/sequence formatting - Wrapping of long lines - Docstrings whose first line was wrapped - A stray backslash in a docstring...
RAPI: Add flag to require authentication
Most RAPI resources do not require authentication for the “GET” method.In some setups it can be desirable to always require authentication.This patch adds a command line parameter to always require it.
Some unrelated minor typos in the “ganeti-rapi” man page are also fixed....
Helper to retrieve access permissions for RAPI resource
Merge similar code into a helper function.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Status change reason support for Reboot
Add support to the Reboot command for specifying the reason for the laststatus change.
Some features are implemented as functions, even if used only once, becausethey will be used by the future patches introducing reason support for all...
New RPC call for node verify in dns-only RPC client
This will be used in node-add.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add the gnt-storage client
Add a new client called 'gnt-storage'.The client interacts with the ExtStorage interface, similarly tothe way gnt-os interacts with the OS interface.
For now, only two commands are supported: 'info' and 'diagnose'.
'diagnose' calculates the node status of each provider on each node,...
Add exclusive_storage parameter to node_info RPC
The flag is passed to the low-level functions that need it, but it's notyet used. Also, RPCs get just a default value for now.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add exclusive_storage parameter to blockdev_create RPC
The flag is passed to the low-level functions that need it, but it's notyet used.
Read watcher pause using RPC, not directly
The master daemon should not directly read files written by the nodedaemon. This patch adds a new RPC to read the watcher pause file andchanges the master code to use it.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Set watcher pause on all nodes
Instead of just setting the watcher pause file, which isn't replicated,RPC is used to set it on all nodes (where possible). This means thatafter an outage of the master node and a following master-failover, thewatcher will still be paused....
Add RPC for setting watcher pause
The watcher pause file should be set/unset on all nodes at once, notonly the master node. For that a new RPC is needed.
Replicate queue drain flag across all master candidates
Until now, the flag was unset on a master failover unless the“$localstatedir/lib/ganeti/queue/drain” file existed.
masterd: Remove duplicate code
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
backend: Rename RunRemoteCommand to RunRestrictedCmd
As per Iustin Pop's suggestion in <20121115131730.GX824@google.com> on<ganeti-devel@googlegroups.com>.
Implement LUNetworkQuery
Summarily list all existing networksSupply detailed info for every existing network - List used/free IPs - List instances with NICs assigned to the corresponding network - List NIC index and IP for the above instances
Implement complementary config methods for retrieving networks....
Basic IP pool management logic
Implement LUs for corresponding opcodes: * LUNetworkAdd: - Check for IP validity - Reserves all necessary IPs - Create new Network config object * LUNetworkRemove: - Checks if connected to any nodegroup - Remove a Network config object...
Add RPC for restricted commands
Expose the functionality to run restricted commands remotely via RPC.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Expose changing job priority via LUXI
A new LUXI request is added, in both Python and Haskell.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'devel-2.6'
Improve logging of new job submissions
This addresses issue 290: when receiving new jobs, logging isincomplete, and we don't have the job ID and/or summarieslogged. Only later, when the job is queried for or being processed, weknow more.
This is not good when troubleshooting, so let's improve the initial...
Update blockdev's "info" at instance rename
Currently, we set "info" metadata on block devices at device creationtime, but we never update it, leading to stale data in case ofinstance renames. This would not be a big problem in case of regularrenames (assuming this is a rare operation), but importing instances...
Remove support for PUT in noded
This takes care of a FIXME; 2.6 already uses the new method, so we'regood during upgrades.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Move a function from backend to ssconf
The “WriteSsconfFiles” function is used to write ssconf files. By movingit we can avoid importing backend into bootstrap. The latter is importedby CLI programs and backend doesn't have much to do with them.
Remove the python confd server side code
In 2.7 we will only support the Haskell version, if enabled.
Since the original hconfd enabling was a bit hack-ish (copying overthe actual installed ganeti-confd, Python version), the Makefile.amchanges are a bit more involved than just the removal of the Python...
Migrate lib/server/*.py from constants to pathutils
File system paths moved from constants to pathutils.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Bump pep8 version to 1.2
Debian Wheezy will ship with this version, and it has many improved checks compared to 0.6, so let's:
- bump version in the docs- silence some new checks that are wrong due to our indent=2 instead of 4- fix lots of errors in the code where the indentation was wrong by 1...
Fix uses of OpPrereqError without code info
A while back, we did cleanup the code and ensured (manually) that useof OpPrereqError includes an errors.ECODE_* field as secondargument. Since we cannot automate the check for this, it turns outthat more and more such usage has crept over the years, including in...
Merge branch 'stable-2.6'
Another small consistency fix with if branches
While looking at the testability of this piece of code, I found anotherconsistency problem. We have two if branches instead of one, withelif's.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix inconsistency in the LUXI protocol w.r.t. args
This inconsistency was found during rebalancing. Hbal failed because,Ganeti couldn't load the opcode. After digging through the cause, aninconsistency with the "args" field in the LUXI protocol was triggered...
Fix DRBD resize code
There are two bugs in the current resize code, affecting mostly DRBD.
First, due to bugs in old DRBD versions (pre 8.0.14), the code currentlycalls `drbdsetup resize' on both the primary or secondary. However,this is actually wrong per current DRBD (from drbdsetup(8)):...
Correct capitalisation of two Luxi calls
Two Luxi calls have inconsistent an name/value mapping (in the Pythoncode):
- REQ_AUTOARCHIVE_JOBS versus AutoArchiveJobs (versus AutoarchiveJobs)- REQ_QUEUE_SET_DRAIN_FLAG versus SetDrainFlag (no Queue)
While these are only a consistency issue, let's fix them so that the...
errors: Add exception for RAPI testing utilities
This exception is raised to abort before actually sending a LUXI call(there is no LUXI server involved in the test). The testing utilitiescatch the exception to report a success (i.e. the code didn't throw...
server.rapi: Factorize RAPI user loading
By moving the code loading RAPI users from a file into a standalonefunction it becomes easier to use the RAPI request handler intests.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
RemoteApiHandler: Add support for mocked LUXI client
This will be used for providing a type-checking RAPI client for tests.
Stop acquiring BGL for LUXI queries
Short description: This fixes an issue whereby masterd would becomeunresponsive on the LUXI socket, leading to client timeouts. While madeworse in 2.5, the underlying issue was already present in 2.4.
Longer description: Until now all LUXI queries would acquire the BGL...
Stop using locks in LUXI "QueryTags"
Also mark it as deprecated in NEWS as normal queries can be usedinstead.
Drop objects.QueryRequest
It was only used in one place and wasn't really necessary.
Merge branch 'devel-2.5'
Add job support to query2 via LUXI
This enables the use of filters through query2 when listing jobs.
cmdlib: Stop forking in LUClusterQuery
While debugging another issue we realized that LUClusterQuery forks.This turned out to be the “platform.architecture” function from thePython library. It uses the “file” command to determine the architectureof the Python binary....
http.server: Move error message formatting to handler class
Like before this patch moves more functionality from the actual serverclass into a separate handler class. At the same time the function ischanged to return both content-type and body instead of relying on a...
noded: Shorter docstring for mlockall(2) executor