History | View | Annotate | Download (11.7 kB)
Implement evacuation mode in hbal
This mode restricts the list of instances to be moved to the instancesliving on the offline (and drained) nodes.
Signed-off-by: Iustin Pop <iustin@google.com>
Split the balancing function in two parts
Currently in the balancing function we do two thing:
- take the decision where to do a new balancing round or not- and actually computing the balancing round
This is not nice, as the two parts are conceptually separate, so this...
Switch the text file format to single-file
This patch changes from the two separate files to a single file, withsections separated by a blank line. Currently only the node and instancedata is accepted, later the cluster tags will be read too via thisformat....
Read cluster tags in the RAPI backend
This also shows them in hbal in verbose mode.
Introduce support for reading the cluster tags
While these are not actually populated from the backends, and all theprograms ignore them, this patch contains the changes in the functiontypes required.
Add a command-line option to filter exclusion tags
Since we don't want all instance tags to be used for exclusion, we add acommand line option to filter on these. Since the iallocator protocolcannot accept command line options, currently it's not possible to...
Allow overriding the field list in -p
The print nodes option can now accept an optional field list tocustomise the output. This is ugly, since the field names do not matchthe header names, but it is at least barely customisable (at runtime).
Add loading and processing of utilisation data
This patch adds loading and processing the utilisation data duringinstance moves. While the data is not yet used, it is correctly modifiedby instance changes between nodes.
hbal has the new ‘-U’ command line argument for this. The format of the...
Allow displaying the instance map in hbal
This is similar to --print-nodes, but with much fewer fields.
Style change: cluster CStats camel-casing
This is again the cs_x to csX name change.
Implement first version of job execution
This not so nice patch adds submit/wait-for-completion functionality inhbal. When enabled via ‘-X’ (only working on Luxi), it executes the jobs(as split into separate jobsets by Cluster.splitJobs) and waits until...
Extend the MoveJob type to hold the instance index
This will be needed in order to generate the proper instance move commands.
Store the instance move in the MoveJobs
This will automatically sort our Ganeti jobs into the independent jobsets, and then we can submit them separately.
Move some more type definitions to Types.hs
hbal: Implement grouping of moves into jobsets
Since moving two instances between different node-quadruples (inst X: A,B → C, D and inst Y: E, F → G, H) can be parallelised by Ganeti, itmakes sense to split the operation list into jobsets whose execution...
Split the exernal data loader out of CLI.hs
Currently the external data loader is in CLI.hs, which makes allprograms that need cli functionality (options, etc.) link against thenetwork modules (most importantly curl). This patch splits thisfunctionality into a new module such that (for example) hail which only...
Split the balancing algorithm in two parts
Currently the computation, recursing part and the IO part (progressupdates) of the balancing main function (iterateDepth) are all in thesame function, which makes it hard to test. This patch moves thedecision/computation part (whether to proceed one more round, whether we...
Implement support for 'cheap' moves only
This patch adds support for cheap (failover/migrate) operations only inthe balancing algorithm and in the hbal command line options.
This allows a very quick balancing (compared to allowing replace-disks)which can be useful as a scheduled operation.
Fix a typo in hbal.hs
Signed-off-by: Guido Trotter <ultrotter@google.com>
Unify the command line options and structures
This patch moves all the command line options and their internalrepresentation into CLI.hs. This means that duplicated options betweenany two binaries are no longer declared twice, and that we no longerneed the two *Option classes.
Add support for luxi backend in CLI/hspace/hbal
This patch changes the backend selection method in CLI to prefer, in order: - a RAPI specification - a Luxi specification - and finally the node/instance files
It also modifies hspace and hbal to provide a ‘-L’ command line option...
Show errors on stderr instead of stdout
Currently many of the exit and warning conditions mistakenly display errormessages on stdout, which makes parsing the output of programs harder. Thispatch attempts to fix such occurrences.
Fix hlint-generated warnings
This big patch cleans up the code per hlint indications. Many removalsof extra parentheses, replacements of concat . map with concabtMap,extra dollar signs, eta reductions, etc. were performed.
The code still compiles and passes a couple of manual tests on sample...
Add a new type for cluster statistics
Currently totalResources returns a 5-tuple of integers. This is not easyto handle, as each change on the return type means that each caller mustbe updated.
This patch adds a new type for cluster stats and uses that instead as...
Add display of more stats in hspace
This patch changes Cluster.totalResources to compute more details aboutthe cluster status, and enhances hspace to display more of these.
Some documentation updates for the new parameters
Add cpu/disk limits in hbal
Fix the various monomorphism warning
In a few places (e.g. tryRead or any printf call) it's a little bit hardto add the correct type signatures, but in the it is possible to fixthese warnings (which can bite one in subtle cases).
Add copyright/license information
This doc-patch adds copyright and license information to (hopefully) allneeded files.
Change the module import hierarchy
This patch makes the Types module a base module, and Node/Instance onesimport it, from the previous (opposite) situation. This will allow inthe future to use newtypes for the index and name types.
Remove the ktn/kti from second half of loading
This removes the return of ktn/kti from Loader.mergeData and associatedfunctions.
Remove most uses of ktn/kti
This patch removes all uses of ktn/kti from the past-loader stages.
Remove some extraneous uses of ktn/kti
Since we have Node/Instance.name, we can now simplify a few constructs.
Some small layout changes
Move common loading sequence in CLI
This patch moves the common loading sequence to CLI, such that hbal/hn1and possible future scripts that take the input from same sources canuse it.
Move checkData from Cluster to Loader
This moves the remaining loading function to Loader (together with itsassociated support functions).
Introduce a class for CLI options
This patch generalizes a little the CLI handling by not passing in aspecial function for help and such, but instead requiring that theoptions object supports some common functionality.
More code reorganizations
This new big patch does a couple of more cleanups in the loading of datachapter: - introduce a Types module that holds most types (except the base Node/Instance/etc.) so that multiple other modules can use these (instead of only Cluster and its users)...
Rework the loader model
This big patch changes the loader model from “string data as commonformat” to actual object structures as common format.
The text loading function move from Cluster.hs to a new Text.hs module,some common functions are moved to a new Loader.hs module, and the...
Convert Cluster.loadData to Result return
This patch changes Cluster.loadData to return a Result, instead ofdirectly the values; this will allow us to return meaningful errorvalues (e.g. when an instances lives on unknown node) rather than simplyabort. Currently the result is always an Ok, the actual signalling of...
Switch from hand-written monads to a real one
This big patch converts from our home-grown monad-like constructs(the Either stuff) to a real, Either-like-but-another-name monad.
We introduce a “Result a” monad, and this allows dropping many of theextra constructs. Hopefully the code is also more clear....
hbal: add a --quiet option
This option is the opposite of the --verbose option, and it allowsdecreasing the verbosity level from the default of one to zero (whichcurrently doesn't show the warning messages for missing disk/memory).
hbal: Simplify the oneline formatting
This patch moves the oneline format into a separate function for easierusage.
hbal: Early exit when we don't have any instances
For clusters with no instances, there is no point in computing eitherthe score or in running the algorithm. In this case, we exit prematurelyand when running in one-line mode we show dummy information.
hbal: Add a new min-score option
This new parameter causes the algorithm to finish (or even not start atall) if we reach/have a score better than it.
Add reading the file names from env vars
This patch adds support for selecting the instance/node file names viatwo environment variables (HTOOLS_NODES, HTOOLS_INSTANCES).
Unfortunately we still have lots of duplicated code, since the optionsare not unified.
Implement error checks for extra arguments
Neither hbal nor hn1 take any arguments beside the options, so if anyare passed is most likely an unintended error. This patch aborts in suchcases.
Implement writing the command list to a script
This patch adds support in hbal for writing the command list to a shellscript, with error checking and allowing for early exit.
hbal: Abort for invalid offline node names
Since it's easy to pass a wrong node name as offline, we should abortinstead of silently ignoring it.
Show the x_mem/i_mem in node list
This patch adds checking of cluster data in the binaries and display ofnode's x_mem/i_mem in the node list.
Split common CLI functionality into a module
This patch moves the common CLI functionality (as much as currentlypossible) into a separate module. This means we only have one parseOptsand that Utils.hs doesn't keep this kind of functions anymore.
Add a separate type for the [(Int, String)] list
This is added for better readability, since this is very often used indeclarations.
Add command line support for offlining nodes
This patch modifies hbal (only, hn1 not yet) for setting nodes offline.
Introduce a namespace for the modules
The modules are moved from the ‘top’ namespace to ‘Ganeti.HTools’, incompliance with standard practices.