Add the predicate of the queue being open
Adding jobs to the queue is only allowed if the queueis not drained.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Provide path to the queue drain file
Since luxid is going to write to the job queue,it needs to honor drains of the queue as well.
Add Luxi Request to pick up a job in the queue
During the transition to the new daemon layout, from step 2onwards, luxid will write to the queue but masterd will triggerthe execution. Therefore, add a new luxi request to tell masterdto pick up a job that has already been written to the queue....
Provide means of locking a file
Two avoid two processes simultaneously accessing the sameon-file structure, like the job queue, file locks are used.Therefore, provide this functionality in Haskell as well.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Provide path to the queue lock file
To avoid several processes accessing the queue at the same time,Ganeti locks the queue via a lock file on disk. Provide the pathto this file.
Provide a function to write jobs to disk
This function writes a (non-archived) job to disk. The filename can be computed from the job id, which is part of the job.
Provide means to allocate new job ids
Add utility functions to allocate new job ids by increasingthe value stored in the serial file. As this function isused in a multi-threaded program, synchronize access overan internal lock.
Add function to get master candidates from configuration
With luxi daemon taking over part of the job queue management, it willalso be responsible for replicating the queue to all master candidates.Therefore, add a function to extract the list of master candidates from...
Support RPC asking to replicate part of the job queue
To be able to replicate the job queue, in particular the serial,luxid needs to be able to send the jobqueue_update RPC. So addits definition.
Provide function to obtain the unique element of a list
This version of 'the' properly lives in the 'Result' monad,as opposed to traditional one calling 'error'. The reasonwhy it is 'Bad' that not precisely one element is returnedis given as argument....
Provide method to read job serial number
This methods allows reading the maximal job serial number fromdisk.
Provide convenience function to create Job from op-codes
This function handles the pure part of generating a job,i.e., assuming the job id already assigned and not settingtime stamps.
Add function to resolve dependencies in meta op code
When queueing many jobs, the dependencies between them need tobe resolved with the knowledge of their respective job id.Lift the computation of the absolute dependency to the levelof MetaOpCodes.
Add function to compute the absolute id of a dependency
SubmitManyJobs also accepts jobs with dependencies given asrelative ids. Together with the absolute id of the job, onceassigned, the dependency can be resolved. Add a function doingthis computation....
Provide a convenience method to optain a QueuedOpCode
When generating jobs from sequences of op-codes, it is necessaryto wrap op-codes into queued form.
Add utility function tryAndLogIOError
This function allows to use 'IO a' objects in a safeway, using the 'try' function; the outcome is reportedas a 'Result'. IOErrors are logged and the result is'Bad', while in the case of no errors, a result-yielding...
Provide utility to atomically write a file
To keep our on-file data consistent at any moment, we changefile contents by atomically replacing the file with a new one.
Fix getNodeRole
In the configuration, the master node is now givenby its uuid. Therefore, compare the uuid and not thename to find out if a given node is the master.
Merge branch 'stable-2.10' into master
Fix documentation
Fix documentation in constants containing values in seconds.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Hs2Py constants: 'hvsParameterTypes' and 'hvsParameters'
Add constants 'hvsParameters' and 'hvsParameterTypes' to the Haskellto Python constant generation.
hsqueeze: support planing for onlining nodes
If the amount of free resources falls below a given threshold,hsqueeze will suggest putting standby nodes back online untilthe minimum of free resources is reached, or all standby nodesare online.
Add an --minimal-resources option
Add a new option, to be used by hsqueeze, to specifythe amount of free resources that has to be on eachnode, in order not to start onlining standby nodes.It is given as a multiple of the standard allocation,as specified by the instance policy....
Text Backend: correctly read data for offline nodes
With standby nodes, simply ignoring the specification ofan offline node is not sufficient any more.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Change default for target resource to 2.0
The target resources, as used by hsqueeze, are supposed tobe strictly higher than the minimal resources. However,keeping minimal resources of less than a single instanceis not a useful reserve.
SimpleRetry on BlockDev.Remove()
Sometimes, upon disk removal, corresponding file descriptorsare kept briefly open by various processes (hypervisor, blkid, etc.).With this patch, we retry several times before raising the appropriateerror, thus making disk removal more robust against those corner cases....
Add possibility to compress to OpInstanceCreate
OpInstanceCreate now supports the 'compress' option. It allows to enablecompression during instance imports.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Support import with compressed instance moves
Support compressing instance data while sending it to the target node oninstance imports.
Add local compression to OpBackupExport
OpBackupExport is extended by a compress parameter. This parameter(either 'none' or 'gzip') controls if instance disks are compressedbefore being sent over the network to the destination node.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Add possibility to compress to OpInstanceMove
OpInstanceMove now supports the 'compress' option. It allows to enablecompression for intra-cluster instance moves.
Remove trailing whitespace
Remove trailing whitespace from OpCodes.hs and OpParams.hs.
Add hsqueeze planning for compression
Add a new htool, hsqueeze, for dynamic power management. Thiscommit only implements the first useful part: plan (but notexecuting) taking nodes offline nodes still keeping within theresource limit.
Provide --target-resources option
Add a new option, to be used by hsqueeze, to specify thetarget free resources on each node. It is given as a multipleof the standard allocation, as specified in the instance policy.
If a configuration file can't be loaded, show which one
So far the error message only says it can't be loaded, but not what fileit is.
Signed-off-by: Petr Pudlak <pudlak@google.com>Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Refactor `getAllConsoleParams` to be more readable
Instead of creating a lot of lists and then zipping them, just workon every item one at a time. It makes the code much more readable.This changes the ordering of "effects", but the "effects" here are just...
Hs2Py constants: add 'hvcDefaults'
Add 'hvcDefaults' to the Haskell to Python constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Hs2Py constants: add 'diskDtDefaults'
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Hs2Py constants: add 'value*' and 'niccDefaults'
Hs2Py constants: add 'ipolicyDefaults'
Add constant 'ipolicyDefaults' to the Haskell to Python constantgeneration.
Hs2Py constants: add 'mond' latest version
Add 'mond' latest version constant to the Haskell to Python constantgeneration.
Hs2Py constants: add 'sshDaemonKeyfiles'
Add constant 'sshDaemonKeyfiles' to the Haskell to Python constantgeneration.
Hs2Py constants: add 'diskLdDefaults'
Add constant 'diskLdDefaults' to the Haskell to Python constantgeneration.
Hs2Py constants: add 'ndcDefaults'
Add constant 'ndcDefaults' to the Haskell to Python constantgeneration.
Hs2Py constants: add 'becDefaults'
Add constant 'becDefaults' to the Haskell to Python constantgeneration. An instance of 'PyValue' is added to the type 'PyValueEx'in order to allow polymorphic types, namely, 'Map's.
Remove 'Show' from 'PyValue' class
Remove 'Show' constraint from the 'PyValue' class which, even thoughit allowed to simplify the definition of instances of this class forprimitive types, it made it more complicated to define instances forother types....
Hs2Py constants: add 'drbdValidBarrierOpt'
Add constant 'drbdValidBarrierOpt' to the Haskell to Python constantgeneration.
Hs2Py constants: add 'dtsLvm'
Add constant 'dtsLvm' to the Haskell to Python constant generation.
Hs2Py constants: remove useless constants
Remove unused constants from Python because they were only used byconstants which are currently generated, and move Python comment tothe Haskell side.
Hs2Py constants: add 'daemonsExtraLogbase'
Add constant 'daemonsExtraLogbase' to the Haskell to Python constantgeneration and update Python reference.
Hotplug: client support
Add --hotplug option. Only used in OpInstanceSetParams.If this is omitted, modifications become effective after reboot.
Ask user confirmation in case NIC modify + hotplug because it willbe done via removing old NIC (and the corresponding tap) and adding...
Hotplug: objects and constants additions
Add pci slot in NIC/Disk objects. This slot will be used onlyby hypervisor code. Currently only KVM will use it and store ittemporarily in runtime files.Add HOTPLUG_* constants to define device types an hotplug actions....
Merge branch 'stable-2.9' into stable-2.10
Merge branch 'stable-2.8' into stable-2.9
Fix 'hvVncTls' key in 'hvsParameterTypes'
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Optimize instance query speed
The instance queries have a field called console which requires anextra RPC call. This commit allows extra parameters to be passed onto the query logic, which allows us to determine when to make theextra call.
Signed-off-by: Hrvoje Ribicic <riba@google.com>...
Tests for InstanceInfo retrieval added
As the retrieval of instance information is a tricky operation whichaffects the way the instance status is calculated, a few tests havebeen added to ensure its behaviour remains constant.
Add the console field
This patch adds the final console field, using the RPC calls of theprevious commit.
As this requires another RPC call, the live data retrieval functionsof the instance queries are modified and refactored slightly.
Haskell RPCs can now differ in parameters
The Haskell interface allowed multi-node RPCs to be executed inparallel, but with the same parameters every time. This restrictionis now lifted by providing a function call accepting pairs of nodesand parameters....
Add a console information RPC call
As the instance queries need console information, and the informationis retrieved through python classes that should not be ported yet, anRPC call supplying the information has been added. Some tests as well.
Add the last leftover fields
A few fields were missing from earlier work. The timestamp fields,the pnode.group.uuid field, and the custom nic parameters.Just the console remains.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Add aliases of existing instance fields
For legacy reasons, some instance fields are aliased. The aliases havethe same definition but appear under different names. Support for suchfields has been added to Query/Common.hsAs aliases can cause somewhat harder to find bugs, a new test was...
Rename Fillable fields to Indexed fields
The naming used in the previous commits did not reflect the use ofthe fields well, and clashed with the concept of fillable parameters.Thus some renaming was done, with slight documentation improvements.
Add instance network fields
This commit adds the instance network fields. Some of these fields arealso present for every NIC, requiring the reuse of functions from theprevious disk instance fields commit.
Add instance disk fields
The instance disk fields have been added. As these are the firstfields to be duplicated, some helper functions are present to supporteasier specification of fields which have to be duplicated for everyentry in an array of configuration objects, and numbered accordingly....
Add instance parameter fields
This patch adds instance parameter fields to the Haskell instancequeries. The fields retrieve maps of parameters related to e.g.hypervisors, or operating systems.Every instance can have parameters different from the defaults,...
Add further fields to instance queries
This commit adds some more fields that are simple to retrieve, asthey are properties of instances recorded in the configuration.
Add the first instance query live fields
This patch adds live fields which can be implemented with the existingRPC calls. Some fields require additional call types and will be addedlater. Most of them can be added at this point.The status field is especially interesting, as it depends not on the...
Add a new field type - FieldConfigRuntime
In the instance queries, some fields require both config and runtimeinformation. This patch adds a new field type for this purpose.
Move hypervisor fetches from Node.hs to Common.hs
As patches that follow will need the functionality inGaneti.Query.Node, this patch moves it to the common query file.
Enable 'AllInstancesInfo' RPC call in Haskell
The AllInstancesInfo call appears to have been implemented but neverused. The commit makes it usable by modifying the parameter type andserialization.
Unify all live calls which return no data
With the (current) instance query, there are three queries which do notrequire live data: instance, network, and group queries.The collectors and runtime types for these have been unified.
Initial changes for adding instance queries
Add basic instance querying capabilities to the Haskell codebase.Several fields are present, more to come.
Document the generic query function
The way the generic query works was not documented anywhere. Forgreater readability, add a short explanation to the genericQueryfunction.
Fix path for serial file
It is actually located inside the queue directory.
Make 'gnt-node list-storage' use default storage type
Currently, when no explicit storage type is given, the'gnt-node list-storage' command defaults to file storagewhether or not file storage is enabled on the clusteror not. This patch fixes it by defaulting to the default...
Fix several search path configure options
Fix several search path configure options to properly handleseparators.
Correct properties of the cluster's file storage dir
This patch does:- Rename the haskell opcode parameter 'pGlobalFileStorageDir' and 'pGlobalSharedfileStorageDir' to 'pClusterFileStorageDir' and 'pClusterSharedfileStorageDir', respectively, because...
Merge branch 'stable-2.9' into master
Make information about GNU ln available to programs
Propagate the information on whether we have the GNU version of lnto all programs by making it available as a constant.
Hs2Py constants: hail, opcode, and SSH
Add hail, opcode, and SSH related constants to the Haskell to Python constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Hs2Py constants: node daemon setup
Add node daemon setup related constants to the Haskell to Python constant generation.
Hs2Py constants: opcodes
Add opcode related constants to the Haskell to Python constant generation.
Hs2Py constants: improve auto repair
Fix auto repair related constants to use the Haskell type 'AutoRepairResult' instead of simple 'String' values.
Fix import order
Fix import order to be alphabetically sorted.
Fix 'AutoConf' import
Fix 'AutoConf' import to be qualified.
Hs2Py constants: dynamic devices
Add dynamic devices related constants to the Haskell to Python constant generation.
Hs2Py constants: SSCONF
Add SSCONF related constants to the Haskell to Python constant generation.
Hs2Py constants: 'defaultEnabledHypervisor'
Add 'defaultEnabledHypervisor' to the Haskell to Python constant generation.
Hs2Py constants: 'hvcGlobals' and 'ndcGlobals'
Add 'hvcGlobals' and 'ndcGlobals' to the Haskell to Python constant generation.
Hs2Py constants: 'ispecsMinmaxDefaults'
Add 'ispecsMinmaxDefaults' to the Haskell to Python constant generation.
Hs2Py constants: assorted constants
Add some assorted constants to the Haskell to Python constant generation.
Hs2Py constants: 'maxUdpDataSize'
Add 'maxUdpDataSize' constant to the Haskell to Python constant generation.
Hs2Py constants: UID pool
Add UID pool related constants to the Haskell to Python constant generation.
Add assorted constants to the Haskell to Python constant generation.
Hs2py constants: query resources
Add query resources related constants to the Haskell to Python constant generation.
Hs2Py constants: HT related constants
Add HT related constants to the Haskell to Python constant generation.