History | View | Annotate | Download (15.6 kB)
Merge branch 'stable-0.2'
hbal: return meaningful exit code for job failures
Currently, LUXI job failures only display a warning message, whilestill returning a success exit code. We change hbal to returntrue/false from within execJobSet/runJobSet, and add a wrapper forsimpler code....
Change the Node.group attribute
Currently, the Node.group attribute is the UUID of the group, as untilrecently Ganeti didn't export the node group properties. Since it doesso now, we make the following changes (again apologies for a bigpatch):
- we change the group attribute to be an index, similar to the way an...
hbal: display the group name in the multi-group case
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
Text.hs: serialize cluster tags when writing data
This is the complement to the reading part. Now the live-test workscorrectly against clusters with configured exclusion tags.
Rework the data loader pipelines to read groups
This (invasive) patch changes all the loader pipelines to read the nodegroups data from the cluster, via the various backends. It is invasiveas it needs coordinated changes across all the loaders.
Note that the new group data is not used, just returned....
hbal: implement handling of multi-group clusters
On a single-group cluster, we proceed as before. On multi-groupclusters, we require selection of the desired group (currently via UUIDonly).
hbal: change handling of signal
Currently, hbal does a one-two signal handling, where the first signalcauses graceful termination, and the second one an immediate on (eitherSIGINT or SIGTERM can be used, interchangeably). However, this poses atiming problem: if two programs want to send a graceful termination...
hbal: implement user-friendly termination requests
Currently, hbal will abort immediately when requested (^C, or SIGINT,etc.). This is not nice, since then the already started jobs need to betracked manually.
This patch adds a signal handler for SIGINT and SIGTERM, which will, the...
Use the mingain options in the balancing algorithm
Also adds them in hbal.
Implement cluster state saving in hbal
Also move the LUXI execution (-X) to the end, after all the outputmessages are printed. No good in waiting for the messages for a longwhile, especially as they are not up-to-date stats after the jobexecution, just an estimation of what the state will be.
hbal: print short names in steps list
This was a regression from the name handling changes, as we startedusing the original names for the solution list (which is not designedfor parsing/feeding back into ganeti).
Fix handling of offline options and short names
This needs to be abstracted in a separate function, but in the meantimewe fix the issue in both places.
Signed-off-by: Iustin Pop <iustin@google.com>
Rename the job status constants
The rename is done such that we match Ganeti's own constants.
Change the Luxi operations structure
Currently, we define the LuxiOp type as a simple enumeration, and leavethe arguments structure to the users of the Ganeti.Luxi module. This issuboptimal for a couple of reasons: first, we decouple the operationtype from operation arguments, and that means we don't use the type...
Stop modifying names for internal computations
Currently the name used internally is modified and holds the shortenedname of the nodes/instances. This has caused issues before, since wealways have to strip the suffix from input data and reapply it if we...
Accept both full and short names in CLI
This patch introduces some new functionality in the base Element typeand in Container which supports searching for all 'known' names of anelement, such that both short and full names are accept for variousoptions like '-O' and '--excluded-instances'.
A number of small fixes from hlint
Fix unused-do-binds for ghc 6.12
GHC 6.12 has some new warnings, which are valid in most cases except(IMHO) printf usage.
Fix unused imports for ghc 6.12
GHC 6.12 has become more picky about unused imports, so we need toremove/tighten some of them.
Enable hbal to use the new command line option
Implement evacuation mode in hbal
This mode restricts the list of instances to be moved to the instancesliving on the offline (and drained) nodes.
Split the balancing function in two parts
Currently in the balancing function we do two thing:
- take the decision where to do a new balancing round or not- and actually computing the balancing round
This is not nice, as the two parts are conceptually separate, so this...
Switch the text file format to single-file
This patch changes from the two separate files to a single file, withsections separated by a blank line. Currently only the node and instancedata is accepted, later the cluster tags will be read too via thisformat....
Read cluster tags in the RAPI backend
This also shows them in hbal in verbose mode.
Introduce support for reading the cluster tags
While these are not actually populated from the backends, and all theprograms ignore them, this patch contains the changes in the functiontypes required.
Add a command-line option to filter exclusion tags
Since we don't want all instance tags to be used for exclusion, we add acommand line option to filter on these. Since the iallocator protocolcannot accept command line options, currently it's not possible to...
Allow overriding the field list in -p
The print nodes option can now accept an optional field list tocustomise the output. This is ugly, since the field names do not matchthe header names, but it is at least barely customisable (at runtime).
Add loading and processing of utilisation data
This patch adds loading and processing the utilisation data duringinstance moves. While the data is not yet used, it is correctly modifiedby instance changes between nodes.
hbal has the new ‘-U’ command line argument for this. The format of the...
Allow displaying the instance map in hbal
This is similar to --print-nodes, but with much fewer fields.
Style change: cluster CStats camel-casing
This is again the cs_x to csX name change.
Implement first version of job execution
This not so nice patch adds submit/wait-for-completion functionality inhbal. When enabled via ‘-X’ (only working on Luxi), it executes the jobs(as split into separate jobsets by Cluster.splitJobs) and waits until...
Extend the MoveJob type to hold the instance index
This will be needed in order to generate the proper instance move commands.
Store the instance move in the MoveJobs
This will automatically sort our Ganeti jobs into the independent jobsets, and then we can submit them separately.
Move some more type definitions to Types.hs
hbal: Implement grouping of moves into jobsets
Since moving two instances between different node-quadruples (inst X: A,B → C, D and inst Y: E, F → G, H) can be parallelised by Ganeti, itmakes sense to split the operation list into jobsets whose execution...
Split the exernal data loader out of CLI.hs
Currently the external data loader is in CLI.hs, which makes allprograms that need cli functionality (options, etc.) link against thenetwork modules (most importantly curl). This patch splits thisfunctionality into a new module such that (for example) hail which only...
Split the balancing algorithm in two parts
Currently the computation, recursing part and the IO part (progressupdates) of the balancing main function (iterateDepth) are all in thesame function, which makes it hard to test. This patch moves thedecision/computation part (whether to proceed one more round, whether we...
Implement support for 'cheap' moves only
This patch adds support for cheap (failover/migrate) operations only inthe balancing algorithm and in the hbal command line options.
This allows a very quick balancing (compared to allowing replace-disks)which can be useful as a scheduled operation.
Fix a typo in hbal.hs
Signed-off-by: Guido Trotter <ultrotter@google.com>
Unify the command line options and structures
This patch moves all the command line options and their internalrepresentation into CLI.hs. This means that duplicated options betweenany two binaries are no longer declared twice, and that we no longerneed the two *Option classes.
Add support for luxi backend in CLI/hspace/hbal
This patch changes the backend selection method in CLI to prefer, in order: - a RAPI specification - a Luxi specification - and finally the node/instance files
It also modifies hspace and hbal to provide a ‘-L’ command line option...
Show errors on stderr instead of stdout
Currently many of the exit and warning conditions mistakenly display errormessages on stdout, which makes parsing the output of programs harder. Thispatch attempts to fix such occurrences.
Fix hlint-generated warnings
This big patch cleans up the code per hlint indications. Many removalsof extra parentheses, replacements of concat . map with concabtMap,extra dollar signs, eta reductions, etc. were performed.
The code still compiles and passes a couple of manual tests on sample...
Add a new type for cluster statistics
Currently totalResources returns a 5-tuple of integers. This is not easyto handle, as each change on the return type means that each caller mustbe updated.
This patch adds a new type for cluster stats and uses that instead as...
Add display of more stats in hspace
This patch changes Cluster.totalResources to compute more details aboutthe cluster status, and enhances hspace to display more of these.
Some documentation updates for the new parameters
Add cpu/disk limits in hbal
Fix the various monomorphism warning
In a few places (e.g. tryRead or any printf call) it's a little bit hardto add the correct type signatures, but in the it is possible to fixthese warnings (which can bite one in subtle cases).
Add copyright/license information
This doc-patch adds copyright and license information to (hopefully) allneeded files.
Change the module import hierarchy
This patch makes the Types module a base module, and Node/Instance onesimport it, from the previous (opposite) situation. This will allow inthe future to use newtypes for the index and name types.
Remove the ktn/kti from second half of loading
This removes the return of ktn/kti from Loader.mergeData and associatedfunctions.
Remove most uses of ktn/kti
This patch removes all uses of ktn/kti from the past-loader stages.
Remove some extraneous uses of ktn/kti
Since we have Node/Instance.name, we can now simplify a few constructs.
Some small layout changes
Move common loading sequence in CLI
This patch moves the common loading sequence to CLI, such that hbal/hn1and possible future scripts that take the input from same sources canuse it.
Move checkData from Cluster to Loader
This moves the remaining loading function to Loader (together with itsassociated support functions).
Introduce a class for CLI options
This patch generalizes a little the CLI handling by not passing in aspecial function for help and such, but instead requiring that theoptions object supports some common functionality.
More code reorganizations
This new big patch does a couple of more cleanups in the loading of datachapter: - introduce a Types module that holds most types (except the base Node/Instance/etc.) so that multiple other modules can use these (instead of only Cluster and its users)...
Rework the loader model
This big patch changes the loader model from “string data as commonformat” to actual object structures as common format.
The text loading function move from Cluster.hs to a new Text.hs module,some common functions are moved to a new Loader.hs module, and the...
Convert Cluster.loadData to Result return
This patch changes Cluster.loadData to return a Result, instead ofdirectly the values; this will allow us to return meaningful errorvalues (e.g. when an instances lives on unknown node) rather than simplyabort. Currently the result is always an Ok, the actual signalling of...
Switch from hand-written monads to a real one
This big patch converts from our home-grown monad-like constructs(the Either stuff) to a real, Either-like-but-another-name monad.
We introduce a “Result a” monad, and this allows dropping many of theextra constructs. Hopefully the code is also more clear....
hbal: add a --quiet option
This option is the opposite of the --verbose option, and it allowsdecreasing the verbosity level from the default of one to zero (whichcurrently doesn't show the warning messages for missing disk/memory).
hbal: Simplify the oneline formatting
This patch moves the oneline format into a separate function for easierusage.
hbal: Early exit when we don't have any instances
For clusters with no instances, there is no point in computing eitherthe score or in running the algorithm. In this case, we exit prematurelyand when running in one-line mode we show dummy information.
hbal: Add a new min-score option
This new parameter causes the algorithm to finish (or even not start atall) if we reach/have a score better than it.
Add reading the file names from env vars
This patch adds support for selecting the instance/node file names viatwo environment variables (HTOOLS_NODES, HTOOLS_INSTANCES).
Unfortunately we still have lots of duplicated code, since the optionsare not unified.
Implement error checks for extra arguments
Neither hbal nor hn1 take any arguments beside the options, so if anyare passed is most likely an unintended error. This patch aborts in suchcases.
Implement writing the command list to a script
This patch adds support in hbal for writing the command list to a shellscript, with error checking and allowing for early exit.
hbal: Abort for invalid offline node names
Since it's easy to pass a wrong node name as offline, we should abortinstead of silently ignoring it.
Show the x_mem/i_mem in node list
This patch adds checking of cluster data in the binaries and display ofnode's x_mem/i_mem in the node list.
Split common CLI functionality into a module
This patch moves the common CLI functionality (as much as currentlypossible) into a separate module. This means we only have one parseOptsand that Utils.hs doesn't keep this kind of functions anymore.
Add a separate type for the [(Int, String)] list
This is added for better readability, since this is very often used indeclarations.
Add command line support for offlining nodes
This patch modifies hbal (only, hn1 not yet) for setting nodes offline.
Introduce a namespace for the modules
The modules are moved from the ‘top’ namespace to ‘Ganeti.HTools’, incompliance with standard practices.