Add more thorough check for terminal color support to QA
It uses `tput` to check if the current terminal has enough colors.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Add tests for hsqueeze commands
In the examples, pick a node that certainly has to be powered on or off,respectively, and verify that the power command is there, as wellan appropriate migration command in the right position.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
hsqueeze: add option to show or save commands
Add an option to hsqueeze to show, or save in a file, the commandsthat have to be carried out.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
hsqueeze: when balancing also keep the move sequence
In hsqueeze, when computing the balancing sequence, alsoremember the sequence of moves that lead there.
Add function to get the moves between two configurations
Add a function that, given two adjacent cluster configurations ofa balancing sequence, computes the moves that led from the firstto the second configuration.
In the list of involved nodes, drop "no secondary"
When grouping moves into jobs, a new job set is started, if the newmove involves a node also touched by a previous move. When computingthe list of involved nodes, the new primary and secondary nodes of the...
Move saving of a command list to CLI
Move the function that saves a list of a command in a fileto CLI.hs. In this way, it is reusable by other htools.
Merge branch 'stable-2.10' into master
Merge branch 'stable-2.9' into stable-2.10
Add NodeGroup to InstanceConsoleInfoParams
Before, calls to `gnt-instance list -o console` with an instance on anode with a custom SSH port failed because of missing groupconfiguration. This patch fixes the problem.
Signed-off-by: Petr Pudlak <pudlak@google.com>...
Add a simple QA test for a group with a custom SSH port
The test is turned on by setting `"group-custom-ssh-port": true` inthe config.
Unlike `"ssh-port": ...` in "options", this is a single test thatdoesn't affect the rest of the tests.
Export `NodeAdd` and `NodeRemove` from qa_node
This is needed for testing SSH ports so that we can remove a node froma group, add it to a group with a custom SSH port, and then undo thechange.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Allow QA node acquisition and group creation using `with`
For acquiring nodes use `with AcquireManyNodes(num): ...`. The nodeswill be released automatically.For creating a new group use `with NewGroupCtx() as group: ...`. Thegroup will be removed automatically....
Update NEWS with information about custom SSH ports
Update custom SSH ports config when downgrading to 2.10
If SSH ports other than 22 are configured, abort the config downgrade,as the operation would certainly fail. Otherwise just removendparams/ssh_port from the configuration.
Include ssh_port in the man page
Added to the other node parameters in ganeti.rst.
Update QA to test custom SSH ports
This patch adds the `ssh-port` option. If set to a non-standard port,the QA script sets up the default node group with this port, and beforerunning tests it adds `iptable` rules to all nodes so that the nodessee each other's SSH servers as running on this port....
Use custom SSH ports for other operations
In particular copyfile and renew-crypto.
Use configured SSH ports when connecting to a console
This is accomplished by passing the corresponding node group tohv_*.py. Tests for hv_*.py that call GetInstanceConsole updated.
Update unit tests for custom SSH ports
The unit tests now include the configurations of SSH ports for nodegroups/nodes.
Use custom SSH ports in node groups when working with nodes
Calling `gnt-instance console` with a custom SSH port doesn't work yet.
Add "ndp/ssh_port" node group configuration parameter
The parameter is added to Haskell sources, from which the correspondingPython code is generated.
Add a method retrieving `NodeGroup` objects by group name
Also add a method for the retrieval of "ndparams" (node parameters)of groups, not just nodes.
group queries: test niceSort and remove FIXME
In an effort to get rid of the python queries soon, thispatch fixes a FIXME of the group queries regarding themissing testing of niceSort in this context. Due to thelack of actually weirdly named hostnames, this patch...
Don't allow optional node parameters
Ganeti does not support optional fields in parameters(hypervisor-params, disk-params, etc.). OpenVSwitch related nodeparameters were the exception to this rule, which caused numerousproblems related to import/export and (de-)serialization....
Move OVS node parameters to the right place
OpenVSwitch node parameters were passed as separate command line flags.This was not only inconsistent with all other parameters(hypervisor-params, disk-params, ...), but also with the documentedbehavior in the man pages....
Make NIC VLAN queryable
Add an additional query field which makes it possible to query forVLANs of instance NICs.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Pass VLAN parameter correctly in moves
Fix the assignment of a wrong value to the VLAN and name parameters ofNICs during instance moves.
Use constant instead of raw string
Use the appropriate constant instead of a raw string.
Fix instance info
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Add QA tests for instance shutdown detection
Add QA tests for instance shutdown detection with Xen PVM and HVMhypervisors.
Fix class methods in KVM hypervisor
Several methods in KVM hypervisor behave like 'classmethods', however,they did not have the '@classmethod' annotation. This patch adds'@classmethod' annotations to several of these methods with the goalof making '_StopInstance' available to the QA. This step is necessary...
Update tests
Update hypervisor unit tests.
Watcher cleans user shutdown domains
Update watcher to destroy shutdown domains when it detects that adomain is in the state 'USER_down'. This requires adding logicsimilar to that of restart instance, that is tracking the number ofcleanup attempts in the watcher's state and give up if a given number...
Haskell instance queries report 'USER_down'
Update instance queries on the Haskell codebase to report 'USER_down',similarly to the Python instance queries.
Python instance queries report 'USER_down'
Update instance queries on the Python codebase to report 'USER_down'in the situation where the user has shutdown a Xen domain but Ganetistill marks this domain as 'ADMIN_up' in its configuration file.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Prevent instance start when user down
Prevent starting a given instance when that instance has been shutdownby the user, given that the instance must be first properly shutdown.
Xen handle domain shutdown
Update Xen backend to properly recognize when a domain has beenshutdown by the user and to properly cleanup a shutdown domain whenGaneti requests Xen to stop this domain.
Add instance state 'USER_down'
Add instance state 'USER_down' which is a state used in reporting onlyand it represents the situation in which the user has shutdown theinstance but Ganeti's configuration still has this instance marked as'ADMIN_up'.
Preserve Xen domain after shutdown
Change Xen domain configuration to preserve the domain state aftershutdown, as opposed to automatic reboot, which is the defaultbehaviour.
Add Haskell hypervisor instance state
Add 'InstanceState' datatype which is the Haskell counterpart of thePython type 'HvInstanceState'.
Add Python hypervisor instance state
Add 'HvInstanceState' which represents the state of an instance(either 'running' or 'shutdown') and fix the remaining hypervisorbackends to return a value of this type. Before this patch, eachbackend was returning their own value, some were returning strings,...
Fix typo in documentation
Merge 'hs2py-constants' into 'hs2py'
Merge code of 'hs2py-constants' into 'hs2py', which requires addingflag '--constants', and merge target 'src/hs2py-constants' into'src/hs2py' in 'Makefile.am', also updating dependencies andvariables.
Add flag '--opcodes' to 'hs2py'
This is a transitional step before merging 'hs2py-constants' and'hs2py'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
man gnt-network: indicate that --network is mandatory
Despite the fact that the '--network' parameter of'gnt-network add' is mandatory, it is listed in the manpage in parenthesis, which suggests that it is optional.Fixing this to avoid user confusion....
Add missing Constructor for SetParamsMods
Disks and nics can not only be addressed by indices, but alsoby name. Hence add a constructor for this case as well, to befaithful to the python world.
Ignore hlint warning "Error: Too strict if" in Server.hs
A previous patch [229da00] added an annotation for ignoring the warning,but to the middle of a function, which doesn't compile. This patch movesthe annotation to the end of the function to correct the problem....
This warning appears only in newer versions of hlint (mine was v1.8.43),and in this case it's reported incorrectly. The arguments to "showJSON" have different types, therefore it's not possible to move "showJSON" in...
Sort design docs alphabetically
Sort design docs alphabetically in Makefile.am and fix whitespace.
Missing design docs from distribution
Fixes issue 609.
Bump revision for 2.9.1
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Update NEWS and schedule release for 2.9.1
Now that issue 608 fixed, schedule a new release datefor 2.9.1.
Fixing rapi documentation wrt to storage types
This fixes a FIXME in the rapi documentation. The list ofstorage types that can be queries for space information waschecked against the complete storage type set and notagainst the set of storage types that actually support...
Fix retrieval of xen command in class method
This patch fixes issue 608. When introducing theconfigurability of the xen toolstack in commit8ef418bb92, the hypervisor api was accidentally changedin a way that let to this error in KVM.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Document incompatible change of luxi
With the change to luxid handling job queue requests, wealso inherited the parsing semantics of luxid, i.e., arequest is only considered syntactically correct, if allparts are. Document that we decided to consider this change...
Adapt TestJobSubmission to new semantics of SubmitManyJobs
Historically, Ganeti would partially handle syntactically incorrectluxi requests, if the outer structure was that of SubmitManyJobsrequest. Moreover, this behavior was verified by a test. We changed to...
Update future upgrade plans
Now that mandatory opcode fields are mandatory also on disk,document that, once we use soft drains during upgrades, the diskrepresentation of the job queue needs to be updated by cfgupgradeas well.
Add a constructor to BaseOpCode
In order to guarantee all mandatory slots to be present, adda custom constructor to BaseOpCode adding those fields withdefault value, instead of inheriting the constructor fromoutils.ValidatedSlots.
FillDict: do not use exceptions for flow control
While it's generally better to beg for forgiveness than ask forpermission in Python, it's not a good idea to rely on exceptions tomodel control flow, especially if the resulting code is simpler.
Signed-off-by: Santi Raffa <rsanti@google.com>...
Remove hardcoded references to File, SharedFile templates
DTS_FILEBASED is a constant that exists and this commit makes surethat it is used whenever sensible, rather than resorting to hardcodingthe pair of templates in very many files.
Fix docstring for ganeti.storage.filestorage_unittest.py
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Make masterd call luxid to write jobs to queue
As the responsibility for writing the job queue changedto luxid, make masterd call to luxid when it is necessaryto write a job.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Set reason trail for derived opcodes
As the base parameters also includes the reason trail,set it, if not set already, together with the otherbase parameters in the case of an opcode expanding toa list of opcodes.
Make luxid handle SubmitManyJobs
Handle this request by writing the jobs to thequeue and inform masterd; masterd will then alsodistribute the jobs to all master candidates.
Make luxid handle SubmitJob
As luxid is to take over responsibility for the job queue,handle this request by writing the job to the queue and theninforming masterd; masterd will also distribute the job toall master candidates.
Add the predicate of the queue being open
Adding jobs to the queue is only allowed if the queueis not drained.
Provide path to the queue drain file
Since luxid is going to write to the job queue,it needs to honor drains of the queue as well.
Add Luxi Request to pick up a job in the queue
During the transition to the new daemon layout, from step 2onwards, luxid will write to the queue but masterd will triggerthe execution. Therefore, add a new luxi request to tell masterdto pick up a job that has already been written to the queue....
Move Haskell constants to proper module
Move Haskell constants from module 'Ganeti.HsConstants', which was atransitional module part of the Haskell to Python constant generationinfrastructure, to module 'Ganeti.Constants'.
Tear down Py2Hs constant infrastructure
Tear down Python to Haskell constant conversion infrastructure, whichincludes eliminating the autotool 'convert-constants' and the Haskellmodule, namely 'Ganeti.PyConstants', which held the convertedcontants.
Factor out functionality to pickup a job from the queue
The current restart procedure for masterd includes functionalityto pick up a job from the queue and restart it, if it hasn't beenstarted before. Move this functionality into a separate functionto be able to have the enqueuing be done by luxid....
Provide means of locking a file
Two avoid two processes simultaneously accessing the sameon-file structure, like the job queue, file locks are used.Therefore, provide this functionality in Haskell as well.
Provide path to the queue lock file
To avoid several processes accessing the queue at the same time,Ganeti locks the queue via a lock file on disk. Provide the pathto this file.
Provide a function to write jobs to disk
This function writes a (non-archived) job to disk. The filename can be computed from the job id, which is part of the job.
Hs2Py constants: additional module jstore
Add constants from additional modules ('ganeti.jstore') to the Haskellto Python constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Hs2Py constants: additional module errors
Add constants from module 'ganeti.errors' to the Haskell to Pythonconstant generation.
Provide means to allocate new job ids
Add utility functions to allocate new job ids by increasingthe value stored in the serial file. As this function isused in a multi-threaded program, synchronize access overan internal lock.
Add function to get master candidates from configuration
With luxi daemon taking over part of the job queue management, it willalso be responsible for replicating the queue to all master candidates.Therefore, add a function to extract the list of master candidates from...
Support RPC asking to replicate part of the job queue
To be able to replicate the job queue, in particular the serial,luxid needs to be able to send the jobqueue_update RPC. So addits definition.
Provide function to obtain the unique element of a list
This version of 'the' properly lives in the 'Result' monad,as opposed to traditional one calling 'error'. The reasonwhy it is 'Bad' that not precisely one element is returnedis given as argument....
Provide method to read job serial number
This methods allows reading the maximal job serial number fromdisk.
Provide convenience function to create Job from op-codes
This function handles the pure part of generating a job,i.e., assuming the job id already assigned and not settingtime stamps.
Add function to resolve dependencies in meta op code
When queueing many jobs, the dependencies between them need tobe resolved with the knowledge of their respective job id.Lift the computation of the absolute dependency to the levelof MetaOpCodes.
Add function to compute the absolute id of a dependency
SubmitManyJobs also accepts jobs with dependencies given asrelative ids. Together with the absolute id of the job, onceassigned, the dependency can be resolved. Add a function doingthis computation....
Provide a convenience method to optain a QueuedOpCode
When generating jobs from sequences of op-codes, it is necessaryto wrap op-codes into queued form.
Add utility function tryAndLogIOError
This function allows to use 'IO a' objects in a safeway, using the 'try' function; the outcome is reportedas a 'Result'. IOErrors are logged and the result is'Bad', while in the case of no errors, a result-yielding...
Hs2Py constants: additional module qlang
Add constants from additional modules ('ganeti.qlang') to the Haskellto Python constant generation.
Provide utility to atomically write a file
To keep our on-file data consistent at any moment, we changefile contents by atomically replacing the file with a new one.
Hs2Py constants: additional module luxi
Add constants from additional modules ('ganeti.luxi') to the Haskellto Python constant generation.
Remove module 'ganeti.constants' from 'convert-constants'
Given that there are no more constants in 'ganeti.constants' toconvert to Haskell, this module import and conversion logic can beremoved from the autotool 'convert-constants'.
Fix reference to vcs version in query server
Fix reference to vcs version in query server to take its value fromthe Haskell constant in 'Ganeti.Version' instead of using the constantgenerated from Python.
Eliminate unused constants
Use configure constants instead of generated
Replace uses of the generated 'AF_INET*' constants with the constantsin Haskell's 'AutoConf'.
Hs2Py constants: add 'UUID_REGEX'
Add constant 'UUID_REGEX' to the Haskell to Python constantgeneration.
Add 'AF_INET*' constants to Haskell's 'AutoConf'
Add 'AF_INET*' constants to the Haskell 'AutoConf' module, whichrequires extending the 'm4' invocation with additional substitutions,in the 'Makefile' target for 'src/AutoConf.hs'.
Python autotool to print socket constants
Add Python script to autotools which prints socket related constants.These constants are a transitional solution for eliminating the Pythonto Haskell constant generation given that the final solution willrequire a change in the configuration and, therefore, postponed to...
Undo revision bump
Before releasing 2.9.2, we still have issue 608 to fix; ifno release date is set, we still have to be at the lowerversion.
Design document for custom SSH ports
See issue 235. It turned out the feature is a bit more complex than expected,hence this design doc.
Signed-off-by: Petr Pudlak <pudlak@google.com>Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Postpone release of 2.9.1
...until issue 608 is fixed.