Iustin Pop [Sun, 18 Oct 2009 21:39:02 +0000 (06:39 +0900)]
Some small style fixes
Iustin Pop [Sun, 18 Oct 2009 21:17:26 +0000 (06:17 +0900)]
Simplify the cstats initializer
Since all values are initialized to zero, the exact ordering is not
important and thus we can use the positional mode for simpler code.
The patch also adds docstrings to the cstats functions.
Iustin Pop [Sun, 18 Oct 2009 20:39:08 +0000 (05:39 +0900)]
Simplify Cluster.computeMoves
Since we now have an actual type for describing the instance moves
(IMove), it's simpler to convert this into the move description/move
commands, rather than re-computing the move based on initial and final
nodes. This makes the shell commands computation and over-Luxi command
execution use the same method of computation.
Iustin Pop [Sun, 18 Oct 2009 20:18:57 +0000 (05:18 +0900)]
Remove obsolete export
The ‘Placement’ type has been moved to Types.hs but we kept exporting it
from Cluster, which is not needed.
Iustin Pop [Sun, 18 Oct 2009 16:44:13 +0000 (17:44 +0100)]
Generalise the node/instance listing
This patch introduces a generic formatTable function (based on, and
similar to the Ganeti one, but different and more FP in style) and
changes the node and instance listing to it.
The node list (due to the many variables) is still a little bit hackish
unfortunately…
Iustin Pop [Sun, 18 Oct 2009 16:38:26 +0000 (17:38 +0100)]
Fix instance listing for non-redundant case
Iustin Pop [Fri, 16 Oct 2009 14:24:03 +0000 (16:24 +0200)]
Fix two haddoc/happy docstring issues
Iustin Pop [Fri, 16 Oct 2009 14:20:11 +0000 (16:20 +0200)]
Update hbal manpage with details about dynutil
Iustin Pop [Fri, 16 Oct 2009 13:40:31 +0000 (15:40 +0200)]
Start using the utilisation scores in balancing
This enables the per-node load/total available capacity scores to be
used in balancing. Note that the total available capacity is currently
fixed at zero and cannot be changed by the user.
Iustin Pop [Fri, 16 Oct 2009 12:17:01 +0000 (14:17 +0200)]
Add loading and processing of utilisation data
This patch adds loading and processing the utilisation data during
instance moves. While the data is not yet used, it is correctly modified
by instance changes between nodes.
hbal has the new ‘-U’ command line argument for this. The format of the
file is simply instance name and the four stats, space-separated.
Iustin Pop [Fri, 16 Oct 2009 11:54:35 +0000 (13:54 +0200)]
Add an option to input utilisation data
Iustin Pop [Fri, 16 Oct 2009 11:09:55 +0000 (13:09 +0200)]
Merge the Node.setPri and Node.addCpus functions
The latter is only used right after the former in the Loader module, and
we'll need more of this 'update not with the data of this instance'
functionality (which is different than addPri where all information must
be updated).
The patch also changes the signature of Node.setSec (to remain
consistent with setPri).
Iustin Pop [Fri, 16 Oct 2009 07:41:29 +0000 (09:41 +0200)]
Move some utility functions to Utils.hs
These were already duplicate (Text and Simu) and we need tryRead in more places.
Iustin Pop [Fri, 16 Oct 2009 07:09:20 +0000 (09:09 +0200)]
Show the load on nodes in node lists
The strange printf usage is due to some limitation (it seems) in ghc for
very long argument lists. The whole printout should be rewritten later.
Iustin Pop [Thu, 15 Oct 2009 14:05:40 +0000 (16:05 +0200)]
Add initial structure for utilisation balancing
This patch adds the datatypes and modifies the nodes and instance types to have
such attributes. They are not used yet in any way.
Iustin Pop [Thu, 15 Oct 2009 13:53:35 +0000 (15:53 +0200)]
Allow displaying the instance map in hbal
This is similar to --print-nodes, but with much fewer fields.
Iustin Pop [Thu, 15 Oct 2009 08:49:00 +0000 (10:49 +0200)]
Add an explicit export list to Instance.hs
This exports all functions, but it's still good to have.
Iustin Pop [Wed, 14 Oct 2009 14:02:53 +0000 (16:02 +0200)]
More hlint fixes
This makes (for now) the code hlint-clean. This is per se not a huge
gain, but it allows easier tracking of regressions in style later
(one-two new violations are easier to diagnose when not hidden among 20
“known” ones).
Iustin Pop [Wed, 14 Oct 2009 13:43:30 +0000 (15:43 +0200)]
Style change: camel-casing of unittests
Iustin Pop [Wed, 14 Oct 2009 11:26:54 +0000 (13:26 +0200)]
Style change: cluster CStats camel-casing
This is again the cs_x to csX name change.
Iustin Pop [Wed, 14 Oct 2009 11:16:18 +0000 (13:16 +0200)]
Style change: node and instance attributes
This changes from a_b to aB in all node and instance attributes, to
match the standard Haskell style. Also attributes that should have been
camel-cased but weren't were changed (e.g. plist → pList, pnode →
pNode).
Iustin Pop [Wed, 14 Oct 2009 10:45:05 +0000 (12:45 +0200)]
Modify the internals of the detailed CV scores
Before we used a tuple; since we'll need more metrics in the future,
it's simpler to transform this into a list of doubles, whose elements
are handled homogeneously by all the code that needs them.
Iustin Pop [Tue, 13 Oct 2009 10:10:29 +0000 (12:10 +0200)]
Update the hbal manpage w.r.t. job execution
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 13 Oct 2009 09:54:38 +0000 (11:54 +0200)]
Implement first version of job execution
This not so nice patch adds submit/wait-for-completion functionality in
hbal. When enabled via ‘-X’ (only working on Luxi), it executes the jobs
(as split into separate jobsets by Cluster.splitJobs) and waits until
ganeti has finished all of them.
This is a rough version: it waits ad infinitum for completion, has
hardcoded timeouts, etc.
Signed-off-by: Iustin Pop <iustin@google.com>
Iustin Pop [Tue, 13 Oct 2009 09:38:35 +0000 (11:38 +0200)]
Add a command line option for executing jobs
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 13 Oct 2009 09:37:47 +0000 (11:37 +0200)]
Add two specialized Luxi calls
This are higher level wrappers over the basic callMethod.
Signed-off-by: Iustin Pop <iustin@google.com>
Iustin Pop [Tue, 13 Oct 2009 10:19:34 +0000 (12:19 +0200)]
Change iMoveToJob to properly create migrates
The current Cluster.iMoveToJob always creates failovers, which is not
what we want. This simply used the original instances status to select
between these two (this is not optimal by the way, since the status
could have changed in the meantime).
Signed-off-by: Iustin Pop <iustin@google.com>
Iustin Pop [Tue, 13 Oct 2009 08:43:10 +0000 (10:43 +0200)]
Extend the MoveJob type to hold the instance index
This will be needed in order to generate the proper instance move commands.
Signed-off-by: Iustin Pop <iustin@google.com>
Iustin Pop [Tue, 13 Oct 2009 08:01:01 +0000 (10:01 +0200)]
Add a simple module for dealing with Ganeti jobs
This holds for now just the job status definitions and its serialisation
to/from JSON.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 12 Oct 2009 12:20:36 +0000 (14:20 +0200)]
Only update the version file on make dist
Since on regular compiles the version information is not critical, it's
better to only update it on make dist; furthermore, there it is indeed
needed to regenerate it automatically, without requiring the use of make
maintainerclean beforehand.
This patch simply adds a new phony target that forces regeneration as a
prerequisite for the dist rule.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 12 Oct 2009 11:12:45 +0000 (13:12 +0200)]
Fix haddock issues with tuple members
It seems that haddock cannot document tuple members - but arguably, once
one needs to do that, tuples should not be used anymore.
This just moves the comments to the tuple comment.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Wed, 7 Oct 2009 17:28:09 +0000 (18:28 +0100)]
makefile: regenerate version when it changes
If we're building on a different version, the version file needs to be
regenerated.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Wed, 7 Oct 2009 16:08:34 +0000 (17:08 +0100)]
parseNode: don't lookup values in drained nodes
Currently parseNode skips looking for values in offline nodes, but tries
to read them for drained ones. With this patch we treat offline and
drained nodes in the same way (which is compatible with the iallocator
protocol changes introduced in Ganeti 2.0.4, but also doesn't break
retrocompatibility). As a bonus we don't need to calculate the value of
the internal Node "offline" parameter, because we know that the node is
going to be online anyway, if we lookup values (and it was statically
set to offline, before, in the offline case).
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Fri, 2 Oct 2009 15:52:51 +0000 (17:52 +0200)]
Store the instance move in the MoveJobs
This will automatically sort our Ganeti jobs into the independent job
sets, and then we can submit them separately.
Iustin Pop [Fri, 2 Oct 2009 15:48:13 +0000 (17:48 +0200)]
Move some more type definitions to Types.hs
Iustin Pop [Fri, 2 Oct 2009 15:37:37 +0000 (17:37 +0200)]
Add a function converting Placements into Jobs
This converts from htools-specific Placements into Ganeti standard
OpCodes, which will later allow execution via Luxi.
Iustin Pop [Fri, 2 Oct 2009 15:35:48 +0000 (17:35 +0200)]
Add a small implementation of OpCodes
These are just a few opcodes we need for executing instance moves.
Iustin Pop [Fri, 2 Oct 2009 14:52:58 +0000 (16:52 +0200)]
Record the move being performed in a Placement
This will allow a more descriptive output later in the solution list, as
opposed to trying to reconstruct the move from the node indices.
The patch also documents the Placement members.
Iustin Pop [Fri, 2 Oct 2009 12:30:50 +0000 (14:30 +0200)]
Add definitions for more Luxi calls
Iustin Pop [Fri, 2 Oct 2009 12:22:11 +0000 (14:22 +0200)]
Split the Luxi generic parts from the loader
The Luxi loader implements both a generic Ganeti Luxi client and the
loader; it is better if these two are separated. The patch adds a
Ganeti/Luxi.hs (not under HTools!) since that is generic for Ganeti, and
not related necessarily to htools.
Iustin Pop [Wed, 30 Sep 2009 21:09:17 +0000 (23:09 +0200)]
hbal: Implement grouping of moves into jobsets
Since moving two instances between different node-quadruples (inst X: A,
B → C, D and inst Y: E, F → G, H) can be parallelised by Ganeti, it
makes sense to split the operation list into jobsets whose execution
must be serialised, but whose individual jobs can be parallelised.
Of course, this doesn't help, because today a single logical job is for
some cases split in multiple gnt-instance invocations, which (if run
using ‘--submit’) might be wrongly parallelised by Ganeti. So while for
now this is mostly a cosmetic hint to the user, and it just a step for
future improvements.
Iustin Pop [Wed, 30 Sep 2009 00:08:35 +0000 (02:08 +0200)]
Change ExtLoader to only handle I/O errors
Due to the Control.Exception changes between 6.8 and 6.10, using it
portably is difficult. Since we're only interested in handling I/O
errors, we can use prelude's catch and not have to deal with
Control.Exception at all.
The use in Luxi.hs where we just use bracket is fine across the two
versions, so we keep that.
Iustin Pop [Tue, 29 Sep 2009 13:43:25 +0000 (15:43 +0200)]
Brown-paper-bag release fixing haddock issues
Haddock doesn't like pre-processed files (at least not in all versions).
Thus we need to remove the ExtLoader module from the haddock-procesed
file list.
Iustin Pop [Mon, 28 Sep 2009 12:54:13 +0000 (14:54 +0200)]
Update NEWS file for the 0.1.7 release
Iustin Pop [Tue, 29 Sep 2009 11:23:17 +0000 (13:23 +0200)]
Make the test suite return an reasonable exit code
Test.QuickCheck.Batch.runTests doesn't return any error statistics,
which makes the test suite just display errorrs and always exit with
exit code 0. This is not good, since one cannot then actually batch run
tests.
This patch adds a wrapper over Batch.run which also modifies a passed
IORef Int to keep track of how many test failures or test aborts we had.
This makes it easy then to exit with an appropriate exit code.
Iustin Pop [Mon, 28 Sep 2009 14:07:58 +0000 (16:07 +0200)]
Turn on, and fix, more warnings
The Makefile was intented to be -Wall and not simply -W, but I missed
that. This enables more warnings and also enables -Werror (except for
the tests).
Iustin Pop [Mon, 28 Sep 2009 14:01:19 +0000 (16:01 +0200)]
Brown bag fix: invert a test
During testing I used the test inversely to see it triggers correctly,
and committed by mistake the inverted test. Fixing it.
Iustin Pop [Mon, 28 Sep 2009 12:47:18 +0000 (14:47 +0200)]
Add support for building without curl
Since curl is not always needed (e.g. when only using luxi or less
likely file backends only) and is also not always available, it is
useful for building without it. This of course disabled the RAPI
backend.
This patch changes ExtLoader to build with the ‘-cpp’ option which makes
ghc run it through cpp first; and based on whether ‘NO_CURL’ is defined
or not, this toggles RAPI/curl inclusion. The patch also removes the
import of Rapi in QC.hs since it's not actually used in tests.
Invoking make as ‘make HEXTRA=-DNO_CURL’ is enough to trigger the new
build mode.
Iustin Pop [Mon, 28 Sep 2009 12:33:02 +0000 (14:33 +0200)]
Split the exernal data loader out of CLI.hs
Currently the external data loader is in CLI.hs, which makes all
programs that need cli functionality (options, etc.) link against the
network modules (most importantly curl). This patch splits this
functionality into a new module such that (for example) hail which only
deals with file I/O doesn't link against these libraries.
Iustin Pop [Wed, 2 Sep 2009 22:54:26 +0000 (23:54 +0100)]
Fix luxi recvMsg for messages bigger than 4K
This patch fixes a logic bug in luxi that breaks receive of messages
bigger than 4096 bytes. The send message is not impacted as it uses a
different algorithm.
Iustin Pop [Mon, 31 Aug 2009 22:53:24 +0000 (23:53 +0100)]
Test some cases for the cluster score computation
Iustin Pop [Mon, 31 Aug 2009 22:22:47 +0000 (23:22 +0100)]
Add some more instance tests
This include instance text load tests.
Iustin Pop [Sun, 30 Aug 2009 15:55:49 +0000 (17:55 +0200)]
Split the balancing algorithm in two parts
Currently the computation, recursing part and the IO part (progress
updates) of the balancing main function (iterateDepth) are all in the
same function, which makes it hard to test. This patch moves the
decision/computation part (whether to proceed one more round, whether we
got a good result, etc.) into Cluster.hs, and leaves only the iteration
and screen update in hbal.hs.
Iustin Pop [Wed, 26 Aug 2009 08:04:38 +0000 (10:04 +0200)]
Implement support for 'cheap' moves only
This patch adds support for cheap (failover/migrate) operations only in
the balancing algorithm and in the hbal command line options.
This allows a very quick balancing (compared to allowing replace-disks)
which can be useful as a scheduled operation.
Iustin Pop [Wed, 26 Aug 2009 07:45:41 +0000 (09:45 +0200)]
Simplify the wrapIO function
This fixes one warning from hlint.
Iustin Pop [Wed, 26 Aug 2009 07:40:36 +0000 (09:40 +0200)]
Use migrate or failover based on instance state
While we can't guarantee that the instance will be in the same state by
the time the migrate/failover command will be run, we can at least try
to do the right thing assuming no other changes to the cluster state.
Of course, the right fix would be for Ganeti to accept a
migrate-or-failover opcode that atomically does the move…
Iustin Pop [Wed, 19 Aug 2009 10:51:06 +0000 (12:51 +0200)]
Update NEWS file for the 0.1.6 release
Iustin Pop [Wed, 19 Aug 2009 11:02:52 +0000 (13:02 +0200)]
Improve the error message for command line errors
Instead of using ioError . userError, we format the error ourselves.
This is nicer - no ‘)’ at the end of the output.
Iustin Pop [Tue, 18 Aug 2009 16:07:54 +0000 (18:07 +0200)]
Add a simulated cluster data loader
This is useful especially for hspace, where we might want to simulate a
hypothetical cluster to check allocation beforehand.
Guido Trotter [Thu, 4 Jun 2009 12:55:53 +0000 (13:55 +0100)]
Fix a typo in hbal.hs
Signed-off-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Wed, 15 Jul 2009 17:22:44 +0000 (19:22 +0200)]
CLI: Handle error better
This patch adds an error handler for any exceptions that are raised
during the external data load phase. This can be improved further, but
it's a good start.
Iustin Pop [Wed, 15 Jul 2009 08:31:51 +0000 (10:31 +0200)]
Unify the command line options and structures
This patch moves all the command line options and their internal
representation into CLI.hs. This means that duplicated options between
any two binaries are no longer declared twice, and that we no longer
need the two *Option classes.
Iustin Pop [Tue, 14 Jul 2009 14:17:26 +0000 (16:17 +0200)]
Document the --vcpus option to hspace
Iustin Pop [Tue, 14 Jul 2009 14:15:25 +0000 (16:15 +0200)]
Fix a few hlint errors
Iustin Pop [Tue, 14 Jul 2009 14:01:38 +0000 (16:01 +0200)]
Man page updates
This patch beautifies the man pages for hbal and hspace.
Iustin Pop [Tue, 14 Jul 2009 14:00:48 +0000 (16:00 +0200)]
CLI: Prevent incompatible options to be selected
This patch makes CLI abort if more than one backend is selected.
Iustin Pop [Tue, 14 Jul 2009 13:17:32 +0000 (15:17 +0200)]
Update documentation for the new luxi backend
Iustin Pop [Tue, 14 Jul 2009 13:04:47 +0000 (15:04 +0200)]
Add support for luxi backend in CLI/hspace/hbal
This patch changes the backend selection method in CLI to prefer, in order:
- a RAPI specification
- a Luxi specification
- and finally the node/instance files
It also modifies hspace and hbal to provide a ‘-L’ command line option
for enabling Luxi.
Iustin Pop [Tue, 14 Jul 2009 12:01:51 +0000 (14:01 +0200)]
Initial commit of the luxi backend
This patch adds a luxi backend that allows direct query of the master
daemon on the local node. This patch doesn't enable the backend to be
used.
There are a couple of things still missing in the implementation:
- we don't have a master timeout in reads and writes, only a
per-recv/send-call one
- the failure reporting is not as good as it could be
Iustin Pop [Tue, 14 Jul 2009 09:58:22 +0000 (11:58 +0200)]
Introduce timeout in RAPI queries
The patch adds two constants in Types.hs for connect and query timeout,
then modifies Rapi.hs to use them as the connect and general curl
timeout.
Rapi could be improved more, as currently we wait double the total
timeout due to not aborting early in case the node queries failed.
Iustin Pop [Thu, 9 Jul 2009 13:54:51 +0000 (15:54 +0200)]
Update NEWS file for the 0.1.5 release
This is basically a hspace release, so the changelog is small.
Iustin Pop [Thu, 9 Jul 2009 13:58:23 +0000 (15:58 +0200)]
Fix a haddoc issue
Iustin Pop [Thu, 9 Jul 2009 13:16:46 +0000 (15:16 +0200)]
hspace: fix failure handling of tryAlloc results
Currently hspace doesn't handle failures from tryAlloc correctly; this
patch changes the iterateDepth function in hspace to return a Result (…)
so that errors can be propagated correctly.
The patch also changes one output key to be more clear and a typo in
Cluster.hs
Iustin Pop [Thu, 9 Jul 2009 12:44:24 +0000 (14:44 +0200)]
Change the tryAlloc/tryReloc workflow
Currently, the tryAlloc and tryReloc function return a list with all the
results, both failures and successes. This is fine for hail, which does
one round of allocations, but is not so good for hspace, which does
iterative rounds; since at each (successful) step we only take the best
solution, it means that we're using lots of heap space to compute and
store node lists which are thrown away at the end of the step.
This patch changes these two functions and their callers in hail/hspace
to only return the best solution, and error/success counters. This
allows hspace to run in a much smaller space, and reduces GC cost
greatly.
Overall, it is a cleanup, as hail/hspace did a lot of work to chose this
best solution, whereas now it's automatically promoted within
Cluster.concatAllocs.
Iustin Pop [Wed, 8 Jul 2009 17:11:45 +0000 (19:11 +0200)]
Simplify the Cluster.tryAlloc structures
Currently the tryAlloc function calls the
allocateOnSingle/allocateOnPair and the builds a new tuple with those
functions's result plus the new node list. This is however suboptimal
in two respects:
- the new nodes added are the 'old' versions of the respective nodes,
so even though we don't use more than their names, it's logically
broken
- we do an extra unpack/repack of the result, while we could simply
pass it through if allocateOnX returned the correct result
This patch makes the allocateOnX functions return the node list too and
also removes them and applyMove from the export list, as these are only
used within Cluster.hs.
Iustin Pop [Wed, 8 Jul 2009 16:38:12 +0000 (18:38 +0200)]
Slight change to the internal allocation results
Currently the Cluster.AllocSolution type is defined as a list of
‘(OpResult Node.list, …)’ and the results for applyMove are defined as
‘(OpResult Node.List, …)’. Both these means that the failure/success
indication is hidden in the first elements of this tuple, which makes is
harder to add other elements depending on the success/failure (like the
score for the new node list).
This patch moves the OpResult to outside of the tuple, in effect making
all these ‘OpResult (…)’ which makes the internal tuple elements
consistent. The patch is affecting the applyMove, tryAlloc and
tryRealloc functions, and only briefly the hail and hspace programs.
Iustin Pop [Wed, 8 Jul 2009 16:25:36 +0000 (18:25 +0200)]
Add a 'tags' makefile target
This uses hasktags for building emacs TAGS.
Iustin Pop [Wed, 8 Jul 2009 14:34:46 +0000 (16:34 +0200)]
hspace: switch output to shell-script format
This (big) patch changes the output of hspace from text-format
(separated by ‘: ’) to a shell-snippet, in ‘key=value’ format.
This will allow sourcing the output or parsing it via awk/sed/etc.
Iustin Pop [Wed, 8 Jul 2009 09:49:17 +0000 (11:49 +0200)]
hspace: move instance count and score into CStats
Currently the instance count and cluster score are separated from the
other initial/final phase stats, even though they are very similar. This
patch moves computation of these two into totalResources/CStats and
removes special printing in hspace.
Iustin Pop [Tue, 7 Jul 2009 16:10:18 +0000 (18:10 +0200)]
Fix unittests
The recent OpResult and CPU values additions broke unittests.
Iustin Pop [Tue, 7 Jul 2009 09:09:09 +0000 (11:09 +0200)]
Export more stats in hspace
This patch changes Cluster.totalResources to compute more resources and
prints them in hspace.
Iustin Pop [Tue, 7 Jul 2009 08:32:52 +0000 (10:32 +0200)]
Show errors on stderr instead of stdout
Currently many of the exit and warning conditions mistakenly display error
messages on stdout, which makes parsing the output of programs harder. This
patch attempts to fix such occurrences.
Iustin Pop [Tue, 7 Jul 2009 08:01:30 +0000 (10:01 +0200)]
Fix score calculation to work with empty clusters
Currently the cluster score calculation includes an offline instance
percentage, expressed as “offline inst / (offline + online inst)”, which
results in NaN for empty clusters. This patch changes the calculation
such that clusters with no offline instances (this criteria includes
completely empty clusters) get a score of zero on this component of the
total score.
Iustin Pop [Mon, 6 Jul 2009 15:58:14 +0000 (17:58 +0200)]
hspace: convert N1 error exit into FailN1 result
Currently hspace exits with an error if the cluster is not N+1 compliant
at the beginning of the run. This patch changes hspace such that this
condition is instead treated as a zero-allocation-possible, FailN1 mode.
This allows the same stats to be reported in this case as in the normal
case.
Iustin Pop [Mon, 6 Jul 2009 14:46:51 +0000 (16:46 +0200)]
Some docstring updates
Iustin Pop [Mon, 6 Jul 2009 14:40:01 +0000 (16:40 +0200)]
hspace: add display of instance spec
This is mostly for user-friendliness in the default mode, when we don't
specify the instance parameters.
Iustin Pop [Mon, 6 Jul 2009 22:20:41 +0000 (00:20 +0200)]
Optimize the Utils.stdDev function
This patch optimizes the stdDev function in two respects:
- first, we don't do sum . map which builds an intermediate list, but
instead use a fold over the list to build incrementally the sum;
this should reduce both the time and space characteristics, as we
have fewer objects created
- second, we move from “a ^ 2” to “a * a” as the latter has a much
simpler implementation and thus a higher performance
Since the ‘square’ function is obsoleted by the above the patch also
removes it.
Iustin Pop [Mon, 6 Jul 2009 21:24:14 +0000 (23:24 +0200)]
Take the foldl out of Loader.fixNodes
Currently Loader.fixNodes is foldl' with a complicated function. It
makes more sense to take foldl' out of this function (and put it into
the caller) and let fixNodes be only this internal function.
Iustin Pop [Mon, 6 Jul 2009 21:13:15 +0000 (23:13 +0200)]
Simplify Cluster.computeMoves
This patch changes the function Cluster.computeMoves to use guards and a
couple of subexpressions in order to greatly simplify it.
Iustin Pop [Mon, 6 Jul 2009 20:50:18 +0000 (22:50 +0200)]
Fix hlint-generated warnings
This big patch cleans up the code per hlint indications. Many removals
of extra parentheses, replacements of concat . map with concabtMap,
extra dollar signs, eta reductions, etc. were performed.
The code still compiles and passes a couple of manual tests on sample
files. The individual changes are also small enough as to be visually
easy to confirm.
Iustin Pop [Sun, 5 Jul 2009 15:56:10 +0000 (18:56 +0300)]
Add computation of the failure reason in hspace
This patch enhances hspace to report why the allocation sequence
stopped, both in absolute error count and for the top reason.
Iustin Pop [Sun, 5 Jul 2009 13:09:43 +0000 (16:09 +0300)]
Return correct failure data from Node.add*
This patch alters the Node.addPri/addSec to return correct failure data.
It removes the computeFailN1 function from the module as that used to
combine both mem and disk checks in the same function and thus the real
error reason cannot be deduced when using it.
Iustin Pop [Sun, 5 Jul 2009 12:53:40 +0000 (15:53 +0300)]
Introduce a new type for allocation results
Currently the allocation/move operations workflow return ‘Maybe a’,
which is very convenient but loses all details about the failure mode.
This patch introduces a new data type which encodes the specific failure
mode. It is not yet used correctly (e.g. all node operations result in
FailN1), but the workflow is updated. Most of the changes are in the
hail/hspace tools, the library code required only trivial adjustments.
Iustin Pop [Sun, 5 Jul 2009 12:21:56 +0000 (15:21 +0300)]
Remove hn1 and related code
hn1 was deprecated for a while and this patch removes it altogether. The
support code in Cluster.hs is also removed.
Iustin Pop [Fri, 3 Jul 2009 19:40:35 +0000 (22:40 +0300)]
Display two more stats in hspace
This adds two new stats - sum of reserved ram and disk.
Iustin Pop [Fri, 3 Jul 2009 18:37:51 +0000 (21:37 +0300)]
Fix totalResources avail disk computation
This uses the newly-added Node.availDisk to compute the actual available
disk correctl, and display the total allocatable disk in hspace.
Iustin Pop [Fri, 3 Jul 2009 17:22:06 +0000 (20:22 +0300)]
Add an availDisk node function
This function returns the amount of available disk, which depends on
whether a low disk limit has been configured or not and on the free disk
space of the node.
Iustin Pop [Fri, 3 Jul 2009 17:02:48 +0000 (20:02 +0300)]
Add two new autocomputed vars to Nodes
Currently we track the max disk usage/max vcpus as percentages, however
sometimes it's easier to check against minimum free disk or maximum
number of cpus, as units instead of percentages.
This patch adds two new variables, lo_dsk, hi_cpu, which are recomputed
whenever we change the m_dsk/m_cpu 'master' values.
Iustin Pop [Thu, 2 Jul 2009 21:50:50 +0000 (23:50 +0200)]
Add a new type for cluster statistics
Currently totalResources returns a 5-tuple of integers. This is not easy
to handle, as each change on the return type means that each caller must
be updated.
This patch adds a new type for cluster stats and uses that instead as
its return type. This simplifies its callers while complicating only a
little the computation of the stats.
Iustin Pop [Thu, 2 Jul 2009 10:43:33 +0000 (12:43 +0200)]
Enhance hspace resource display
The display of cluster resources is extracted into a separate function
and enhanced to display more stats.