Instance queries: remove opcodes and LU
Removes the remains of the instance queries.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Export and network queries: remove opcodes and LUs
Removes the remains of the export (aka backup) and networkqueries.
Group queries: remove opcodes and LUs
Removes the remains of the group query code.
Node queries: remove opcodes and LUs
Removes the remains of the node query code.
Remove opcodes from haskell tests
Clears the haskell tests from the query opcodes that are nolonger needed.
Make burnin use luxi queries
Burnin was still using some op queries. With this patch,they are now replaced by luxi queries.
Remove op queries from masterd
Masterd still had some 'if' branches for the queries thatused to be op queries, but are now luxi queries. Thispatch removes them.
Remove instance query python code
This patch removes the python code for the instancequeries. So far, it replaces it by 'NotImplemented'exceptions. In a later patch of this series, theremaining part is remove completely.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Make watcher use queries properly
The watcher did not yet use the query client to makequeries. This patch fixes that.
Adjust console behavior wrt to operation state
Unlike its cli counterpart, the rapi console operation didnot properly take into account whether the instance isactually running. This patch fixes this problem.
User query client necessary for instance queries
There were some places left when code querying for instanceinformation did not use the query client yet.
Switch to Haskell for group queries
This patch removes the group query implementationin python in order to use the new Haskell implementation.
Use query client when neccessary for group queries
This patch makes code use the query client for groupqueries where necessary in order to remove the pythonqueries soon.
Switch to haskell for export (aka backup) queries
This patch removes the python implementation of export(aka backup) queries. So far, it is replaced by'NotImplemented' exceptions, but later in this seriesit will be replaced completely.
Switch to Haskell for network queries
This patch removes the python implementation of networkqueries and replaces it with 'NotImplemented' exceptions.It will be removed completely once all queries areswitched to Haskell.
Disable node query code
This patch removes the python query implementation fornodes. So far, the code is replaced by 'NotImplemented'exceptions, because the overall structure of query classescan be removed more easily at once when all queryimplementations are ready to be removed....
masterd: implement query via luxi
The master daemon so far still did queries via the pythonimplementation. This patch implements that it uses thehaskell implementation and removes the node queries fromthe list of OP-queriable entities.
Make watcher use query clients
So far, the watcher was still issuing node queriesvia the python implementation. This patche switches tothe haskell implementation.
Move GetClient() to runtime module
Soon both, the cli and the masterd, will use a luxiclient to run queries. The method to obtain and configurethe luxi client with the right address can be reused fromcli to masterd here. Therefore, this patch moves the...
Implement 'QueryInstances' call in Haskell luxi server
While the command line uses the generic 'Query' call,rapi calls 'QueryInstances'. 'QueryInstances' so farwas not fully implemented in the Haskell implementationof the luxi server. This was discovered when trying to...
Fix bug regarding node UUID in haskell node queries
When moving from python to haskell node queries, a bugwas discovered where a node's UUID was mistakenlycompared to a node's name. This indirectly caused thecluster epo operation to fail, because it was not...
Use a query client for node queries where necessary
To remove the node query's python implementation soon, thispatch makes all users of the queries call the haskellimplementation instead of the python one.
Remove --enable-split-query option
Switching from python to haskell queries, this patchremoves the option to dis/enable the haskell queriesat configure time.
Add Xen paused state as a Ganeti running state
The QA test 'TestClusterBurnin' pauses instances and the Xen statedomain 'paused' was not being handled properly by Ganeti because itwas not being classified as either a running or shutdown state. Thispatch adds the 'paused' state to the set of running states by...
Fix wrong expectation in hsqueeze shelltest
The shelltests expected the wrong position of the -f optionas well. Fix this.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
hsqueeze: fix position of option in gnt-node power
hsqueeze can produce a shell script with the commands to squeezethe cluster; in the script, fix the position of the '-f' optionin the 'gnt-node power' command.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
build_chroot: hard-code the version of blaze-builder
The newest version does not build on debian squeeze, so avoidit beeing pulled in as a dependency.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Jose Lopes <jabolopes@google.com>
Fix broken TAGS generation
This file is used for development to point to code definitions. Apparently thishas been broken since Februrary [cc40185]. The improved version fixes theproblem and uses Exuberant Ctags <http://ctags.sourceforge.net/>. It uses...
Add patching QA configuration files on buildbots
Add missing spindles paramter to idisk
When spindles where added to Ganeti, apparently it was forgottento add the parameter to the Haskell data type as well. Do this now.
Parse spindles as integer
The disk parameter "spindles", if present, has to be an integer.Enforce this already when parsing the command line, instead ofpassing around the string representation of that integer.
Add an IPv6 check for the QA test of custom SSH ports
If any of the nodes involved would be reached through IPv6, the test isskipped. This is because the required testing setup using "iptables" issuiteble only for IPv4 and doesn't work for IPv6 connections....
hsqueeze: describe option -C in the man page
The option -C was recently added to hsqueeze, but not documented inthe man page. Add the documentation now.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Allow the NIC VLAN to be set to an empty string
The NIC VLAN has previously not been modified via Haskell, causing theINicParams class not to be used. With the recent job queuerefactorings, a modification definition is recorded, and for an emptystring (which is a legal default value) a crash happens. This patch...
Add the aggregate NIC VLAN instance field
Allow the retrieval of the VLANs of all the NICs through nic.vlans.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Add NIC VLAN field retrieval to Haskell queries
The field was added to Python queries in an earlier version, and nowhas to be added to the Haskell queries as well.
Show running times of whole test blocks
Similarly to simple tests, test blocks are now demarcated (in boldface)and measured.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Add more color options to QA
Allow combination of colors, background colors, bold, reverse video,etc.
Add more thorough check for terminal color support to QA
It uses `tput` to check if the current terminal has enough colors.
Add tests for hsqueeze commands
In the examples, pick a node that certainly has to be powered on or off,respectively, and verify that the power command is there, as wellan appropriate migration command in the right position.
hsqueeze: add option to show or save commands
Add an option to hsqueeze to show, or save in a file, the commandsthat have to be carried out.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
hsqueeze: when balancing also keep the move sequence
In hsqueeze, when computing the balancing sequence, alsoremember the sequence of moves that lead there.
Add function to get the moves between two configurations
Add a function that, given two adjacent cluster configurations ofa balancing sequence, computes the moves that led from the firstto the second configuration.
In the list of involved nodes, drop "no secondary"
When grouping moves into jobs, a new job set is started, if the newmove involves a node also touched by a previous move. When computingthe list of involved nodes, the new primary and secondary nodes of the...
Move saving of a command list to CLI
Move the function that saves a list of a command in a fileto CLI.hs. In this way, it is reusable by other htools.
Merge branch 'stable-2.10' into master
Merge branch 'stable-2.9' into stable-2.10
Add NodeGroup to InstanceConsoleInfoParams
Before, calls to `gnt-instance list -o console` with an instance on anode with a custom SSH port failed because of missing groupconfiguration. This patch fixes the problem.
Signed-off-by: Petr Pudlak <pudlak@google.com>...
Add a simple QA test for a group with a custom SSH port
The test is turned on by setting `"group-custom-ssh-port": true` inthe config.
Unlike `"ssh-port": ...` in "options", this is a single test thatdoesn't affect the rest of the tests.
Export `NodeAdd` and `NodeRemove` from qa_node
This is needed for testing SSH ports so that we can remove a node froma group, add it to a group with a custom SSH port, and then undo thechange.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Allow QA node acquisition and group creation using `with`
For acquiring nodes use `with AcquireManyNodes(num): ...`. The nodeswill be released automatically.For creating a new group use `with NewGroupCtx() as group: ...`. Thegroup will be removed automatically....
Update NEWS with information about custom SSH ports
Update custom SSH ports config when downgrading to 2.10
If SSH ports other than 22 are configured, abort the config downgrade,as the operation would certainly fail. Otherwise just removendparams/ssh_port from the configuration.
Include ssh_port in the man page
Added to the other node parameters in ganeti.rst.
Update QA to test custom SSH ports
This patch adds the `ssh-port` option. If set to a non-standard port,the QA script sets up the default node group with this port, and beforerunning tests it adds `iptable` rules to all nodes so that the nodessee each other's SSH servers as running on this port....
Use custom SSH ports for other operations
In particular copyfile and renew-crypto.
Use configured SSH ports when connecting to a console
This is accomplished by passing the corresponding node group tohv_*.py. Tests for hv_*.py that call GetInstanceConsole updated.
Update unit tests for custom SSH ports
The unit tests now include the configurations of SSH ports for nodegroups/nodes.
Use custom SSH ports in node groups when working with nodes
Calling `gnt-instance console` with a custom SSH port doesn't work yet.
Add "ndp/ssh_port" node group configuration parameter
The parameter is added to Haskell sources, from which the correspondingPython code is generated.
Add a method retrieving `NodeGroup` objects by group name
Also add a method for the retrieval of "ndparams" (node parameters)of groups, not just nodes.
group queries: test niceSort and remove FIXME
In an effort to get rid of the python queries soon, thispatch fixes a FIXME of the group queries regarding themissing testing of niceSort in this context. Due to thelack of actually weirdly named hostnames, this patch...
Don't allow optional node parameters
Ganeti does not support optional fields in parameters(hypervisor-params, disk-params, etc.). OpenVSwitch related nodeparameters were the exception to this rule, which caused numerousproblems related to import/export and (de-)serialization....
Move OVS node parameters to the right place
OpenVSwitch node parameters were passed as separate command line flags.This was not only inconsistent with all other parameters(hypervisor-params, disk-params, ...), but also with the documentedbehavior in the man pages....
Make NIC VLAN queryable
Add an additional query field which makes it possible to query forVLANs of instance NICs.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Pass VLAN parameter correctly in moves
Fix the assignment of a wrong value to the VLAN and name parameters ofNICs during instance moves.
Use constant instead of raw string
Use the appropriate constant instead of a raw string.
Fix instance info
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Add QA tests for instance shutdown detection
Add QA tests for instance shutdown detection with Xen PVM and HVMhypervisors.
Fix class methods in KVM hypervisor
Several methods in KVM hypervisor behave like 'classmethods', however,they did not have the '@classmethod' annotation. This patch adds'@classmethod' annotations to several of these methods with the goalof making '_StopInstance' available to the QA. This step is necessary...
Update tests
Update hypervisor unit tests.
Watcher cleans user shutdown domains
Update watcher to destroy shutdown domains when it detects that adomain is in the state 'USER_down'. This requires adding logicsimilar to that of restart instance, that is tracking the number ofcleanup attempts in the watcher's state and give up if a given number...
Haskell instance queries report 'USER_down'
Update instance queries on the Haskell codebase to report 'USER_down',similarly to the Python instance queries.
Python instance queries report 'USER_down'
Update instance queries on the Python codebase to report 'USER_down'in the situation where the user has shutdown a Xen domain but Ganetistill marks this domain as 'ADMIN_up' in its configuration file.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Prevent instance start when user down
Prevent starting a given instance when that instance has been shutdownby the user, given that the instance must be first properly shutdown.
Xen handle domain shutdown
Update Xen backend to properly recognize when a domain has beenshutdown by the user and to properly cleanup a shutdown domain whenGaneti requests Xen to stop this domain.
Add instance state 'USER_down'
Add instance state 'USER_down' which is a state used in reporting onlyand it represents the situation in which the user has shutdown theinstance but Ganeti's configuration still has this instance marked as'ADMIN_up'.
Preserve Xen domain after shutdown
Change Xen domain configuration to preserve the domain state aftershutdown, as opposed to automatic reboot, which is the defaultbehaviour.
Add Haskell hypervisor instance state
Add 'InstanceState' datatype which is the Haskell counterpart of thePython type 'HvInstanceState'.
Add Python hypervisor instance state
Add 'HvInstanceState' which represents the state of an instance(either 'running' or 'shutdown') and fix the remaining hypervisorbackends to return a value of this type. Before this patch, eachbackend was returning their own value, some were returning strings,...
Fix typo in documentation
Merge 'hs2py-constants' into 'hs2py'
Merge code of 'hs2py-constants' into 'hs2py', which requires addingflag '--constants', and merge target 'src/hs2py-constants' into'src/hs2py' in 'Makefile.am', also updating dependencies andvariables.
Add flag '--opcodes' to 'hs2py'
This is a transitional step before merging 'hs2py-constants' and'hs2py'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
man gnt-network: indicate that --network is mandatory
Despite the fact that the '--network' parameter of'gnt-network add' is mandatory, it is listed in the manpage in parenthesis, which suggests that it is optional.Fixing this to avoid user confusion....
Add missing Constructor for SetParamsMods
Disks and nics can not only be addressed by indices, but alsoby name. Hence add a constructor for this case as well, to befaithful to the python world.
Ignore hlint warning "Error: Too strict if" in Server.hs
A previous patch [229da00] added an annotation for ignoring the warning,but to the middle of a function, which doesn't compile. This patch movesthe annotation to the end of the function to correct the problem....
This warning appears only in newer versions of hlint (mine was v1.8.43),and in this case it's reported incorrectly. The arguments to "showJSON" have different types, therefore it's not possible to move "showJSON" in...
Sort design docs alphabetically
Sort design docs alphabetically in Makefile.am and fix whitespace.
Missing design docs from distribution
Fixes issue 609.
Bump revision for 2.9.1
Update NEWS and schedule release for 2.9.1
Now that issue 608 fixed, schedule a new release datefor 2.9.1.
Fixing rapi documentation wrt to storage types
This fixes a FIXME in the rapi documentation. The list ofstorage types that can be queries for space information waschecked against the complete storage type set and notagainst the set of storage types that actually support...
Fix retrieval of xen command in class method
This patch fixes issue 608. When introducing theconfigurability of the xen toolstack in commit8ef418bb92, the hypervisor api was accidentally changedin a way that let to this error in KVM.
Document incompatible change of luxi
With the change to luxid handling job queue requests, wealso inherited the parsing semantics of luxid, i.e., arequest is only considered syntactically correct, if allparts are. Document that we decided to consider this change...
Adapt TestJobSubmission to new semantics of SubmitManyJobs
Historically, Ganeti would partially handle syntactically incorrectluxi requests, if the outer structure was that of SubmitManyJobsrequest. Moreover, this behavior was verified by a test. We changed to...
Update future upgrade plans
Now that mandatory opcode fields are mandatory also on disk,document that, once we use soft drains during upgrades, the diskrepresentation of the job queue needs to be updated by cfgupgradeas well.
Add a constructor to BaseOpCode
In order to guarantee all mandatory slots to be present, adda custom constructor to BaseOpCode adding those fields withdefault value, instead of inheriting the constructor fromoutils.ValidatedSlots.
FillDict: do not use exceptions for flow control
While it's generally better to beg for forgiveness than ask forpermission in Python, it's not a good idea to rely on exceptions tomodel control flow, especially if the resulting code is simpler.
Signed-off-by: Santi Raffa <rsanti@google.com>...
Remove hardcoded references to File, SharedFile templates
DTS_FILEBASED is a constant that exists and this commit makes surethat it is used whenever sensible, rather than resorting to hardcodingthe pair of templates in very many files.
Fix docstring for ganeti.storage.filestorage_unittest.py
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>