Implement ChangeJobPriority in luxid
For jobs still queued, we ask the queue to change the priority,and replicate the changed job. For jobs that have already beenstarted, we have to contact the job directly, which, at the moment,means forwarding the request to masterd....
Make safeRenameFile create dirs with defined permissions
If, and only if, safeRenameFile creates a new directory, make sureit has well defined permissions. While there, also optimize for thecommon case. The main use of safeRenameFile is archiving jobs. As...
Merge branch 'stable-2.10' into stable-2.11
Enable network tags in Haskell code
Prior to the creation of the 2.10 branch, network tags were broken, andthe Haskell code introduced there mistakenly accepted this as thedesired functionality. This patch fixes this in a very simple way.
Signed-off-by: Hrvoje Ribicic <riba@google.com>...
Merge branch 'stable-2.9' into stable-2.10
Implement auto-archiving of jobs
As luxid is taking over the handling of the job queue, italso needs to handle the automated archiving of jobs. Herewe replicate the semantics of the current python implementationof archiving as many jobs older than the given time as possible,...
Make ArchiveJob in luxid create the archive, if necessary
As jobs are archived in groups of 10000, creating new subdirectoriesof the archive might be necessary when archiving a job. Use afunction that takes care of this.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
fix off-by-one error in indentation
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Implement ArchiveJob queries in luxid
With luxid taking over the tasks of masterd, archivingjobs also belongs to its responsibilities. As archivinga job affects the global state of the job queue, synchroniseover the queue lock.
luxid: fix detection of master node in node query
Ganeti.Config.getNodeRole would rely on clusterMasterNode returning themaster node name, however clusterMasterNode returns the master node'sUUID. We fix this and a similar issue in Ganeti.Query.Node.nodeFields....
Make configuration available to the scheduler
In this way, scheduling decisions can depend on the configurationof the cluster. At the moment, this is only the maximal numberjobs to be run in parallel, but in the future this will also includejob filters....
Make max_running_jobs queryable
As we have introduced a new cluster parameter, it shouldbe also visible when querying about the cluster configuration.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Implement job cancellation in luxid
As luxid handles the job queue, this daemon is the naturalplace to handle job cancellation. Answering to CancelJob requestsis also necessary for luxid to be feature compliant with masterd,even for command-line requests only....
User shutdown hypervisor parameter
Add user shutdown parameter for KVM. Based on this parameter, decidewhat information to report for a KVM instance, for example,distinguish between 'ADMIN_down' and 'USER_down'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Fix whitespace
Fix whitespace in several modules.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Also consider filter fields for deciding if using live data
If the query fields don't require live data, we use the shortcutand don't request live data. However, we cannot take this shortcutif the fields the filter depends on requires live data.
Make luxid handle SetDrainFlag
Make luxid also handle queries to drain the job queue.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix sign in drain_flag request
The drain flag is set, if the queue is not open.
ssconf: Add Gluster mount directory
This commit adds the gluster storage directory to ssconf (withoutactually using its value just yet).
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Implement fields query for instance
Support the query for the fields available for instances.
Remove the hvsGlobals from instance query fields
...to be consistent with the python implementation.
When interpreting [] as "all fields", sort nicely
When asked for all fields, we promise to return the list of fieldssorted according to niceSort. Keep this promise.
Merge branch 'stable-2.8' into stable-2.9
Fix gnt-network list-tags
Define network tags in haskell part.
This fixes issue 641.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Handle QueryConfigValues
Make luxid handle the QueryConfigValues call providing certainsimple status information about the cluster.
Add a predicate for watcher pause
Add a predicate, in IO, to test whether the watcher ispaused.
Implement SetWatcherPause in luxid
Make luxid handle SetWatcherPause correctly.
Move the generalized IO client from Luxi to UDSServer
No code is changed in this patch (except imports and qualifiers), onlymoved.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Generalize the IO client handling in Luxi
... to be usable for WConfd as well. A daemon handler is encapsulatedinto `Handler` data type, which is then passed to a generic `listener`.
The changes are done in Luxi.hs so that the differences are visible and...
Add the Unix domain socket path to the Server data type
This simplifies code for closing such a socket.
Encapsulate a server socket and its parameters
Instead of passing a bare server socket around, we pass it encapsulatedin a data type together with parameters such as read/write timeouts.
Rename getClient/Server to getLuxiClient/Server
Later they will be split into LUXI-specific and general parts.
Make luxid support WaitForJobChange
Make support the WaitForJobChange, waiting for a job tochange on certain monitored fields.
Make luxid use the JQScheduler
Make luxid use the job scheduler instead of immediatelystarting every received job.
Rename enqueueJobs to startJobs
This reflects better what the method actually does. Later,we will add a job scheduler that will provide a proper enqueuemethod.
Add default_iallocator_params cluster parameter
Add a cluster parameter to hold the iallocator parameters usedby the default instance allocator. Implement the option tomodify config.data, query config.data and upgrade man pages,tests and cfgupgrade tool. The new default_iallocator_params is...
Set the received time stamp for new jobs
Since luxid now handles the job submission requests, it is alsoits responsibility to set the received time stamps. Do this.
Fix retrieval of number of instances of a node
This patch fixes a FIXME to make the retrieval of thenumber of primary and secondary instances share morecommon code.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Use hypervisor / storage information only when requested
So far, the node queries ignored the list of fields andjust requested all available information from the backend.That means, for example if only hypervisor information isrequested, still the storage space calculation is...
Make luxid job submission be defined by replication
When receiving jobs to be submitted, make luxid replicate them to allmaster candidates and then return. The actual execution can be handledasynchronously.
Rename LuxiSocket to MasterSocket
Rename the constants to name the socket to connect masterd,as the name LuxiSocket hints on luxid, which is differentfrom masterd.
Implement 'QueryInstances' call in Haskell luxi server
While the command line uses the generic 'Query' call,rapi calls 'QueryInstances'. 'QueryInstances' so farwas not fully implemented in the Haskell implementationof the luxi server. This was discovered when trying to...
Fix bug regarding node UUID in haskell node queries
When moving from python to haskell node queries, a bugwas discovered where a node's UUID was mistakenlycompared to a node's name. This indirectly caused thecluster epo operation to fail, because it was not...
Add the aggregate NIC VLAN instance field
Allow the retrieval of the VLANs of all the NICs through nic.vlans.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Add NIC VLAN field retrieval to Haskell queries
The field was added to Python queries in an earlier version, and nowhas to be added to the Haskell queries as well.
Merge branch 'stable-2.10' into master
Add NodeGroup to InstanceConsoleInfoParams
Before, calls to `gnt-instance list -o console` with an instance on anode with a custom SSH port failed because of missing groupconfiguration. This patch fixes the problem.
Signed-off-by: Petr Pudlak <pudlak@google.com>...
group queries: test niceSort and remove FIXME
In an effort to get rid of the python queries soon, thispatch fixes a FIXME of the group queries regarding themissing testing of niceSort in this context. Due to thelack of actually weirdly named hostnames, this patch...
Haskell instance queries report 'USER_down'
Update instance queries on the Haskell codebase to report 'USER_down',similarly to the Python instance queries.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Ignore hlint warning "Error: Too strict if" in Server.hs
A previous patch [229da00] added an annotation for ignoring the warning,but to the middle of a function, which doesn't compile. This patch movesthe annotation to the end of the function to correct the problem....
This warning appears only in newer versions of hlint (mine was v1.8.43),and in this case it's reported incorrectly. The arguments to "showJSON" have different types, therefore it's not possible to move "showJSON" in...
Make luxid handle SubmitManyJobs
Handle this request by writing the jobs to thequeue and inform masterd; masterd will then alsodistribute the jobs to all master candidates.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Make luxid handle SubmitJob
As luxid is to take over responsibility for the job queue,handle this request by writing the job to the queue and theninforming masterd; masterd will also distribute the job toall master candidates.
Fix reference to vcs version in query server
Fix reference to vcs version in query server to take its value fromthe Haskell constant in 'Ganeti.Version' instead of using the constantgenerated from Python.
Refactor `getAllConsoleParams` to be more readable
Instead of creating a lot of lists and then zipping them, just workon every item one at a time. It makes the code much more readable.This changes the ordering of "effects", but the "effects" here are just...
Optimize instance query speed
The instance queries have a field called console which requires anextra RPC call. This commit allows extra parameters to be passed onto the query logic, which allows us to determine when to make theextra call.
Tests for InstanceInfo retrieval added
As the retrieval of instance information is a tricky operation whichaffects the way the instance status is calculated, a few tests havebeen added to ensure its behaviour remains constant.
Add the console field
This patch adds the final console field, using the RPC calls of theprevious commit.
As this requires another RPC call, the live data retrieval functionsof the instance queries are modified and refactored slightly.
Add the last leftover fields
A few fields were missing from earlier work. The timestamp fields,the pnode.group.uuid field, and the custom nic parameters.Just the console remains.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Add aliases of existing instance fields
For legacy reasons, some instance fields are aliased. The aliases havethe same definition but appear under different names. Support for suchfields has been added to Query/Common.hsAs aliases can cause somewhat harder to find bugs, a new test was...
Rename Fillable fields to Indexed fields
The naming used in the previous commits did not reflect the use ofthe fields well, and clashed with the concept of fillable parameters.Thus some renaming was done, with slight documentation improvements.
Add instance network fields
This commit adds the instance network fields. Some of these fields arealso present for every NIC, requiring the reuse of functions from theprevious disk instance fields commit.
Add instance disk fields
The instance disk fields have been added. As these are the firstfields to be duplicated, some helper functions are present to supporteasier specification of fields which have to be duplicated for everyentry in an array of configuration objects, and numbered accordingly....
Add instance parameter fields
This patch adds instance parameter fields to the Haskell instancequeries. The fields retrieve maps of parameters related to e.g.hypervisors, or operating systems.Every instance can have parameters different from the defaults,...
Add further fields to instance queries
This commit adds some more fields that are simple to retrieve, asthey are properties of instances recorded in the configuration.
Add the first instance query live fields
This patch adds live fields which can be implemented with the existingRPC calls. Some fields require additional call types and will be addedlater. Most of them can be added at this point.The status field is especially interesting, as it depends not on the...
Add a new field type - FieldConfigRuntime
In the instance queries, some fields require both config and runtimeinformation. This patch adds a new field type for this purpose.
Move hypervisor fetches from Node.hs to Common.hs
As patches that follow will need the functionality inGaneti.Query.Node, this patch moves it to the common query file.
Unify all live calls which return no data
With the (current) instance query, there are three queries which do notrequire live data: instance, network, and group queries.The collectors and runtime types for these have been unified.
Initial changes for adding instance queries
Add basic instance querying capabilities to the Haskell codebase.Several fields are present, more to come.
Document the generic query function
The way the generic query works was not documented anywhere. Forgreater readability, add a short explanation to the genericQueryfunction.
Hs2Py constants: add 'osApiVersions'
Add constant 'osApiVersions' to the Haskell to Python constantgeneration. This patch fixes the type of this constant, by changingfrom 'List' to 'Set', therefore, 'Ganeti.Query.Server' is updatedaccordingly.
Hs2Py constants: add node parameter related constants
Add node parameter related constants to the Haskell to Python constantgeneration. This patch also fixes a type inconsistency betweenHaskell and Python, namely, 'ndsParameterTypes' and'ndsParameterTitles' become 'Map's instead of 'Tuple's, which allows...
Move data types from 'Objects' to 'Types'
Move several Haskell data types from 'Ganeti.Objects' to'Ganeti.Types' in order to be able to generate constants whileavoiding module circularity.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add constructor 'TagKindNetwork' to 'TagKind'
Add missing constructor 'TagKindNetwork' to 'Ganeti.Types.TagKind'which is the counterpart of the Python constant 'TAG_NETWORK'. Thisfixes issue 578.
Merge branch 'stable-2.9'
Update Harep, Query server, and tests
Update Harep, Haskell query server, and tests concerning Luxi andopcodes to reflect the changes to Haskell to Python opcodegeneration. This change is necessary because TagObject is replaced byTagKind and some types in opcodes and parameters changed to be...
Include VCS version in `gnt-cluster version`
Also print the VCS version in the output of `gnt-cluster version`. Thismakes the VCS version also available over RAPI, etc.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Conflicts: src/Ganeti/Utils.hs (trivial)trivial
Fix permission problem related to Issue 477
Commit 91525dee856951ace940c78b6254a1c7344b4803 fixed Issue 477 but broke"gnt-cluster info".
This commit offers a solution to both problems, by changing the permissionof the socket instead of changing the permission the confd process is run...
Rename queryd to luxid
As queryd will, in the future, handle all LUXI request, queue jobs andmost likely perform various other tasks, it is renamed to luxid already.This will safe some headache when upgrading Ganeti installations, as wedon't have to deal with a daemon rename....
Add queryd daemon (split from confd)
queryd is added as a new daemon which handles configuration queries overLUXI. This functionality was removed from confd, which now only queriesover the network.
The queryd user is added to the master group such that it can access...
Extract ConfigReader from Confd/Server.hs
Confd's functionality to watch the Ganeti configuration file isextracted to the ConfigReader module. No functional changes areintroduced.
This extraction makes will enable us to split queryd from confd, asqueryd will have to use the same functionality....
Add timestamps to haskell network query fields
Add timestamp fields to the list of available network query fields inthe Haskell code.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>Reviewed-by: Helga Velroyen <helgav@google.com>
Merge branch 'stable-2.8' into master
Add luxiReqQueryNetworks to LuxiOp
When the QueryNetwork was introduced as a method, apparentlyit was forgotten in the Haskell world. Add it here as well.
Log received message at debug level
At debug level, we can well afford to have a detailed entryfor each message received by a server.
Log RPC errors from inside executeRpcCall
executeRpcCall is the function to be used for executing RPCs, so it makes senseto use it as the single point for logging all thte RPC call errors.
Fixes Issue 293.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Factor out the logRpcErrors function
This function can be useful to multiple RPC calls, therefore it is movedto the file containing the common RPC functions.
Also, it is made more generic by changing its signature.
Prevent silent failure in case of connection problems
While running "gnt-node list", if a query to ConfD fails (especiallybecause of permission problems) it used to just fail silently, with gnt-nodeshowing question marks instead of data.
With this patch, ConfD records the error in its log file, together with a...
Extraction of storage info by type
There was a bug in the node queries. It was assumed thatthe returned storage space information was in a particularorder. With the changes in the storage reporting, thisorder is not that reliable anymore, in particular, the...
Clean up work around for host name filtering
These functions simply served as a work around to expresshost name matching by regular expressions, instead of usingcorrect equality filter on host names that providesthe correct matching already.
Do not handle host queries special
As, since 91c1a265, the equality used for host names alreadyis based on matching, there is no need to use a special functionfor this any more.
Turn 'exclusive_storage' into storage parameter (hs)
This is the haskell implementation of my patch "Extend RPC call'node_info' by storage parameters". It turns the 'exclusivestorage' flag into a storage parameter of the LVM storage types.Besides that, this patch moves some types into the Types.hs....
Export CPUs used by the node OS
They are exported through the LUXI, RAPI, and IAllocator interfaces.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Make NodeInfo (hs) accept arbitrary storage types
So far, the Haskell implementation of NodeInfo justrequests storage information about volume groups.With this patch, storage info for abitrary storagetypes can be requested.
Signed-off-by: Helga Velroyen <helgav@google.com>...