Iustin Pop [Mon, 20 Dec 2010 15:01:04 +0000 (16:01 +0100)]
Simulation backend: read the allocation policy too
This patch moves the allocation policy from hardcoded to be read from
the given specification, and extends the error message for invalid
specifications.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Mon, 20 Dec 2010 14:54:51 +0000 (15:54 +0100)]
Simulation backend: allow multiple node groups
This patch changes the behaviour of the --simulation option to be an
incremental option, where each new use defines a new node group. This
allows simulation of more complex clusters.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Thu, 23 Dec 2010 15:52:10 +0000 (16:52 +0100)]
Merge branch 'stable-0.2'
* stable-0.2:
Move man files to man/ subdirectory
Conflicts (all removed):
man/hail.1
man/hbal.1
man/hscan.1
man/hspace.1
Iustin Pop [Thu, 23 Dec 2010 14:50:18 +0000 (15:50 +0100)]
Move man files to man/ subdirectory
This is just change on the 0.2 branch to synchronize with the master
branch. It allows automated builds to work better across the two
versions.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>
Iustin Pop [Thu, 23 Dec 2010 15:17:54 +0000 (16:17 +0100)]
Merge branch 'stable-0.2'
* devel-0.2:
Update NEWS file for 0.2.8 release
hbal: return meaningful exit code for job failures
Change the balancing function
Iustin Pop [Thu, 23 Dec 2010 13:04:53 +0000 (14:04 +0100)]
Update NEWS file for 0.2.8 release
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>
Iustin Pop [Thu, 23 Dec 2010 10:32:54 +0000 (11:32 +0100)]
hbal: return meaningful exit code for job failures
Currently, LUXI job failures only display a warning message, while
still returning a success exit code. We change hbal to return
true/false from within execJobSet/runJobSet, and add a wrapper for
simpler code.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>
Iustin Pop [Thu, 23 Dec 2010 10:04:05 +0000 (11:04 +0100)]
Change the balancing function
Currently the balancing function is a modified version of the standard
deviation (stddev divided by list length), due to historical reasons.
While this works fine for small clusters, for big clusters it makes
the balancing effect too "weak", and in some cases it refuses to
balance correctly some clusters. It also makes the balancing behaviour
dependant on the cluster size, which is a big no-no.
Therefore we revert to the normal version of standard deviation, and
we also rename the function to reflect what it does. The new version
correctly balances some corner cases that the previous version didn't,
and passes the current balancing unittests.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>
Iustin Pop [Tue, 21 Dec 2010 09:59:22 +0000 (10:59 +0100)]
Move some tiered spec functionality to Cluster.hs
This splits out a bit of code from hspace.hs and moves it into its own
function in Cluster.hs.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Thu, 16 Dec 2010 14:15:44 +0000 (15:15 +0100)]
IAllocator: respect the alloc_policy for groups
This patch changes the allocate mode to respect the alloc_policy for
groups. It does this by changing the sort key from simply the solution
score, to a tuple with two elements: the alloc policy (which is now an
Ord instance) and the solution score. Also, the unallocable groups are
filtered out in the filterMGResults phase.
The patch also slightly enhances the informational message by
including the policy in the group information, to help debugging.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Fri, 17 Dec 2010 12:26:15 +0000 (13:26 +0100)]
hail: allow overriding cluster data from requests
Currently, it's not easy to generate “fake” IAllocator request files
for hail. As such, testing on simulated clusters is hard to do.
To workaround this, we change hail to also take the ‘-t’ and
‘--simulate’ options, so that we can override the cluster data read
from the request. Note that this will not change the request itself
(so for example an evacuation will need to make sure uses the correct
node names), but it's a step forward in testing hail. The other tools
already can use text files which allow for better flexibility.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Thu, 16 Dec 2010 13:59:43 +0000 (14:59 +0100)]
Text: read/write the allocation policy
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Thu, 16 Dec 2010 13:45:54 +0000 (14:45 +0100)]
Luxi: read the allocation policy from the cluster
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Thu, 16 Dec 2010 13:44:07 +0000 (14:44 +0100)]
Rapi: read the allocation policy from the cluster
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Thu, 16 Dec 2010 13:35:28 +0000 (14:35 +0100)]
Implement a JSON instance for AllocPolicy
This will allow reading this attribute via the Rapi/Luxi backends.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Thu, 16 Dec 2010 12:11:53 +0000 (13:11 +0100)]
live-test: support multi-group clusters
Since currently hbal can only work on single groups at a time, we need
to be able to specify the target group when running the live test.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Thu, 16 Dec 2010 12:03:36 +0000 (13:03 +0100)]
Text.hs: serialize cluster tags when writing data
This is the complement to the reading part. Now the live-test works
correctly against clusters with configured exclusion tags.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Thu, 16 Dec 2010 12:02:31 +0000 (13:02 +0100)]
Text.hs: also read cluster tags from the data file
This means that a file with the correct information is as accurate as
the other backends (Luxi, Rapi). Serialization of tags is in the next
patch.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Thu, 16 Dec 2010 11:46:19 +0000 (12:46 +0100)]
Text.hs: change to use sepSplit
The new sepSplit function can split based on empty lines, so we remove
the hackish text splitting from before and simply use sepSplit. This
is needed as the addition of extra sections would have increased the
code linearly, which we don't want :)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Thu, 16 Dec 2010 11:33:24 +0000 (12:33 +0100)]
Generalise the sepSplit function
Currently it works on splitting strings by individual chars, but we
can generalise it to split lists by list elements, which means we can
reuse it later in the Text module for splitting both lists of chars by
'|' or lists of lines by empty newlines. The change also makes the
code cleaner (uses “null xs” instead of string-specific “xs == ""”).
Note: I tried to rewrite this in a nicer, functional style using
unfolds, but I failed to account for the final terminator case
(e.g. ab|cd|) resulting in a valid but empty element.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Thu, 16 Dec 2010 10:10:59 +0000 (11:10 +0100)]
hail: display group names in info messages
This patch switches from the group index to the group name for the
informational messages in the hail results.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Wed, 15 Dec 2010 18:07:57 +0000 (19:07 +0100)]
hbal: display the group name in the multi-group case
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Wed, 15 Dec 2010 17:49:12 +0000 (18:49 +0100)]
Text.hs: also save the group data when serialising
This should have been in the previous patches, but sent separate for
clarity.
The live-test script is updated to read the first node from the
cluster, now that the text files don't start anymore with the node
data.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Wed, 15 Dec 2010 17:18:47 +0000 (18:18 +0100)]
Change the Node.group attribute
Currently, the Node.group attribute is the UUID of the group, as until
recently Ganeti didn't export the node group properties. Since it does
so now, we make the following changes (again apologies for a big
patch):
- we change the group attribute to be an index, similar to the way an
Instance.pnode and snode attributes point to the parent node(s)
- on load, we read the group.uuid attribute and we use that to lookup
the actual group index, from previously-loaded groups info
- this means that we now first read groups, then read nodes using the
group info, and then read instances using the node info
This patch leaves a few functions showing the group index (ugly since
it's htools internal), will be converted later.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Wed, 15 Dec 2010 15:48:03 +0000 (16:48 +0100)]
Rework the data loader pipelines to read groups
This (invasive) patch changes all the loader pipelines to read the node
groups data from the cluster, via the various backends. It is invasive
as it needs coordinated changes across all the loaders.
Note that the new group data is not used, just returned.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Wed, 15 Dec 2010 15:16:11 +0000 (16:16 +0100)]
Add lookupGroup utility function
This will be used in the various backends similar to the lookupNode
function.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Wed, 15 Dec 2010 15:03:16 +0000 (16:03 +0100)]
Add a new Group.hs module describing node groups
This is not yet used by the rest of the code.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Fri, 10 Dec 2010 14:27:34 +0000 (15:27 +0100)]
Add the new OpQueryGroups opcode definition
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Wed, 15 Dec 2010 10:04:37 +0000 (11:04 +0100)]
Update hscolour usage
This patch fixes two issues related to coloured sources generation.
First, recent hscolour has changed the css file (and we need to update
it), but it also can output it at runtime, so there's no need to store
it anymore in the source tree.
Second, the current source generation predates the addition of sources
in Ganeti/ (as opposed to just Ganeti/HTools), and thus we were missing
the sources in that directory. We replace the target file name to
account for different base directories.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Iustin Pop [Thu, 2 Dec 2010 19:20:37 +0000 (19:20 +0000)]
Improve error reporting for small clusters
When doing a two-node allocation on a cluster/group in which only one
node is online, or a one-node allocation without any online nodes, we
didn't show a valid error mesage. The patch changes tryAlloc to "fail
hard" in this case, to make the failure explicit.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Thu, 2 Dec 2010 19:09:06 +0000 (19:09 +0000)]
hail/allocate: implement multi-group support
This is a bit hackish. We add a new function that takes the input data,
splits it into groups, runs the original tryAlloc for each group, and
then chooses the best solution, but adds the log messages from all the
groups, as to give better debugging information. In hail, we just point
to this new function.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Thu, 2 Dec 2010 17:43:46 +0000 (17:43 +0000)]
hail: remove the custom info message generation
Since the solutions are "self-annotated", we can remove the custom code
from hail, and just keep a very small processResults function.
After this change, allocation/failure shows the new detailed
information.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Thu, 2 Dec 2010 16:39:56 +0000 (16:39 +0000)]
Add a 'log' attribute to allocation solutions
And also a couple of functions for describing a given solution; these
will be used in the future instead of the ones currently in hail.
The patch also enhances the description of failure messages.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Thu, 2 Dec 2010 15:16:23 +0000 (15:16 +0000)]
Change AllocSolution from tuple to its own type
Tuples are good for two, three, at most four elements. Beyond that, the
continuous pattern matching and construction/deconstruction becomes
tedious.
Since in the future we'll probably keep more information in the
AllocSolution type, we change it now from a triple to a "real" data
type. We also do some cleanups: adding a real emptyAlloc value, instead
of the previous hardcoded ones, and add some more comments on how we do
the multi-evacuation.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Wed, 1 Dec 2010 13:58:27 +0000 (13:58 +0000)]
Cleanup AllocSolution after AllocElement changes
Since we added the score to AllocElement, we don't need to wrap
AllocElement in yet another tuple, just to attach the cluster score. So
we simplify the AllocSolution type.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Tue, 30 Nov 2010 23:53:57 +0000 (23:53 +0000)]
AllocElement: extend with the cluster score
AllocElement, a type used as a result of allocations, holds the status
of the nodes after the allocation. In most cases, we'll compare this
allocation result with others, to see which allocation decision makes
the most sense. This comparison is done via the cluster score.
However, if we later need to redo this computation, as part of other
comparisons, we'd need to evaluate it again, etc. So it's easier to just
compute the score at the place where we compute the node list in the
initial step.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Wed, 1 Dec 2010 00:10:57 +0000 (00:10 +0000)]
Add two utility functions for the Result type
Actually, this just moves the functions from the QC module to Types, and
removes a duplicate entry from Cluster.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Tue, 30 Nov 2010 17:35:21 +0000 (17:35 +0000)]
Rework the types used during data loading
This improves on the previous change. Currently, the node and instance
lists shipped around during data loading are (again) association lists.
For instances it's not a big issue, but the node list is rewritten
continuously while we assign instances to nodes, and that is very slow.
The code was originally written for small (10-20 node) clusters, and
today with multinodes… :)
Rewriting to use Node.List/Instance.List makes a bit of a messy patch,
but it allows to remove some custom functions for assoc. list
processing, and also some custom unittests.
At the end, the load time is almost halved, and we spend time now just
in the string parsing code (which is, as we know, slow…).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Tue, 30 Nov 2010 16:31:31 +0000 (16:31 +0000)]
Loader functions: move from assoc lists to maps
When loading big clusters, the association lists become a bit slow, so
we'll replace this with a simple Map String Int; the change is trivial
and can be reverted easily, while it brings up a good speedup in the
data loading.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Tue, 30 Nov 2010 16:23:49 +0000 (16:23 +0000)]
Convert some leftovers to NameAssoc
The type alias NameAssoc has been introduced a long time ago, but there
are some few not-yet-converted cases. In preparation for changes to that
type, let's make sure we use it consistently.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Tue, 30 Nov 2010 15:42:14 +0000 (15:42 +0000)]
hbal: implement handling of multi-group clusters
On a single-group cluster, we proceed as before. On multi-group
clusters, we require selection of the desired group (currently via UUID
only).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Tue, 30 Nov 2010 14:53:18 +0000 (14:53 +0000)]
Add Cluster.splitCluster for node groups
This splits a top-level cluster information into the component node
groups. Instance go to the group of their primary node, but otherwise we
don't disallow split instances.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Tue, 30 Nov 2010 12:24:09 +0000 (12:24 +0000)]
Add the man html files to gitignore
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Tue, 30 Nov 2010 12:20:07 +0000 (12:20 +0000)]
Rework Container.hs and improve test coverage
Since some of the functions we export from Container.hs are 1:1
identical to IntMap, we can just export the originals and remove the
wrappers. This reduces the code we need to unittest.
Furthermore, we add two simple unittest for the two non-trivial
functions that we do have in Container.hs.
And finally, we remove the 'remove' function, since it's not used, and
thus bring code coverage very close to 100%.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Mon, 29 Nov 2010 18:37:33 +0000 (18:37 +0000)]
Add new command-line option for group selection
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Mon, 29 Nov 2010 18:17:26 +0000 (18:17 +0000)]
Add two functions for checking cluster consistency
For now, we don't support instances allocated across two groups, and we
will reject such clusters. The isClusterConsistent function will return
a list of inconsistent instances, potentially allowing operation without
touch them (but only the rest).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Mon, 29 Nov 2010 17:15:19 +0000 (17:15 +0000)]
Add function for nodes to (nodgroup, nodes) split
Unittests included. The function will be needed for consistency checks
in the algorithms.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Mon, 29 Nov 2010 16:52:34 +0000 (16:52 +0000)]
Add a type alias for UUIDs
This is to pottentially allow easier changes later.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Sun, 28 Nov 2010 17:02:24 +0000 (17:02 +0000)]
Also build HTML versions of man pages
Iustin Pop [Mon, 22 Nov 2010 16:53:37 +0000 (16:53 +0000)]
RAPI: read the group UUID from the server
This depends on future support from Ganeti (2.4+).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Mon, 22 Nov 2010 16:38:59 +0000 (16:38 +0000)]
IAlloc: read group uuid from the input message
This makes the code incompatible with JSON files from Ganeti pre-2.4.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Mon, 22 Nov 2010 15:33:13 +0000 (15:33 +0000)]
Text: read/save the node group UUID
Compatibility with old text files is kept by using the default UUID if
the file (or even some records) don't have a UUID.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Mon, 22 Nov 2010 15:15:20 +0000 (16:15 +0100)]
Luxi: read the node uuid from the cluster
This makes the code incompatible with Ganeti pre-2.4.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Mon, 22 Nov 2010 15:00:21 +0000 (16:00 +0100)]
Node: add the node group's UUID
This is not used anywhere yet, and the backend are all just adding the
default UUID, not the real one.
The patch also allows displaying the group UUID in the node list.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Mon, 22 Nov 2010 14:58:22 +0000 (15:58 +0100)]
Utils: add a default UUID
This will be used as a placeholder for the cases when we need a UUID
(any UUID), but we don't have one handy.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Tue, 23 Nov 2010 14:45:51 +0000 (14:45 +0000)]
Merge branch 'devel-0.2' into master
Iustin Pop [Mon, 22 Nov 2010 18:50:22 +0000 (18:50 +0000)]
Improve the standard deviation computation
This does just two passes, instead of three, over the list. This reduces
the overall runtime well enough (~25%) in some tests, but it's not
reproducible using profiling, so I don't know how much the function
itself is being sped-up.
Note: this is written via `seq`s, and not BangPatterns. Since it's just
one case, adding BangPatterns just for it wasn't a big gain.
Thanks to Lécz Balázs for the impetus to improve this!
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Mon, 22 Nov 2010 17:15:30 +0000 (17:15 +0000)]
hbal: change handling of signal
Currently, hbal does a one-two signal handling, where the first signal
causes graceful termination, and the second one an immediate on (either
SIGINT or SIGTERM can be used, interchangeably). However, this poses a
timing problem: if two programs want to send a graceful termination
request, they cannot do that without careful coordination.
To fix this, we change to code to handle the signal separately: SIGINT
(^C) sends graceful termination, while SIGTERM sends immediate
termination. This should allow easier controlling of hbal.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>
Iustin Pop [Fri, 19 Nov 2010 11:11:37 +0000 (12:11 +0100)]
Simu loader: move the loading to non-IO code
While we don't actually have IO code in the Simu loader, we do have the
same interface. So we move the code again to a separate parseData
function which is exported.
Iustin Pop [Fri, 19 Nov 2010 08:23:57 +0000 (09:23 +0100)]
Luxi loader: split parsing from loading
Iustin Pop [Sat, 31 Jul 2010 02:55:25 +0000 (22:55 -0400)]
Rapi loader: split parsing from loading
The change is similar to the text loader change.
Iustin Pop [Sat, 31 Jul 2010 02:46:37 +0000 (22:46 -0400)]
Text loader: split parsing from loadData
This change, which will be followed by similar changes in the other
loaders, splits the parsing of the data from the actual loading from
disk. Since the parsing doesn't usually involve IO actions, we will be
able to better test the parsing. The loading becomes a smaller part of
the code and thus inability to test it has a smaller impact.
Iustin Pop [Thu, 11 Nov 2010 11:01:34 +0000 (12:01 +0100)]
Ignore nodes which are not vm_capable
This break compatibility with Ganeti pre-2.3.
Iustin Pop [Tue, 9 Nov 2010 07:51:45 +0000 (08:51 +0100)]
Merge branch 'devel-0.2'
* devel-0.2:
Fix tag exclusion weight
Iustin Pop [Tue, 9 Nov 2010 07:11:05 +0000 (08:11 +0100)]
Fix tag exclusion weight
Currently, the tag exclusion metric has a weight of one, which means
there might be cases where we won't move instances around because it
upsets the cluster metrics. However, we do want to make a higher effort
for cleaning up tag collisions, so we increase the weight to an
empirically-determined value of 2.
Iustin Pop [Tue, 26 Oct 2010 09:34:37 +0000 (11:34 +0200)]
Force UTF-8 locale for pandoc invocation
Pandoc 1.5.x uses the locale information to parse its input files (only
1.5, pre and post version use always UTF-8). Hence we need to enforce a
UTF-8 locale for proper parsing of input files.
Iustin Pop [Mon, 25 Oct 2010 15:38:28 +0000 (17:38 +0200)]
Move from hand-written man pages to RST/pandoc
This simplifies the maintenance of the man pages, and unifies the rst-to-*
converter to pandoc.
Iustin Pop [Fri, 10 Sep 2010 15:10:26 +0000 (17:10 +0200)]
Add design for htools/Ganeti 2.3 sync
This is a work in progress, will be modified along with the progress
of Ganeti 2.3.
Iustin Pop [Thu, 7 Oct 2010 13:04:58 +0000 (15:04 +0200)]
Update NEWS file for 0.2.7 release
Iustin Pop [Thu, 7 Oct 2010 12:42:46 +0000 (14:42 +0200)]
Fix some warnings in unittests
Iustin Pop [Wed, 6 Oct 2010 12:57:27 +0000 (14:57 +0200)]
Add a hack for normalized CPU values in hspace
Currently, the key metrics/tiered spec computations show the virtual cpu
count. However, since we do have a maximum ration Vcpu/Pcpu, we can also
show the “normalized” cpu count, i.e. the equivalent physical cpu count
corresponding to the virtual ones.
Iustin Pop [Wed, 6 Oct 2010 12:56:14 +0000 (14:56 +0200)]
Improve the error message for tiered alloc option
Iustin Pop [Wed, 15 Sep 2010 15:30:24 +0000 (17:30 +0200)]
hbal: implement user-friendly termination requests
Currently, hbal will abort immediately when requested (^C, or SIGINT,
etc.). This is not nice, since then the already started jobs need to be
tracked manually.
This patch adds a signal handler for SIGINT and SIGTERM, which will, the
first time, simply record the shutdown request (and hbal will then exit
once all jobs in the current jobset finish), and at the second request,
will cause an immediate exit.
Iustin Pop [Wed, 1 Sep 2010 18:12:29 +0000 (20:12 +0200)]
Document the gain options in hbal's manpage
Iustin Pop [Wed, 1 Sep 2010 18:07:54 +0000 (20:07 +0200)]
Use the mingain options in the balancing algorithm
Also adds them in hbal.
Iustin Pop [Wed, 1 Sep 2010 16:31:54 +0000 (18:31 +0200)]
Add new CLI options for min gain during balancing
Recent hbal seems to run many steps for small improvements (< 1e-3), so
we should stop early in this case.
We add a new option (-g), that will be used for the minimum gain during
balancing. This check will only become active when the cluster score is
below a threshold (--min-gain-limit), so as to not stop rebalances too
early.
Iustin Pop [Thu, 2 Sep 2010 12:45:07 +0000 (14:45 +0200)]
Makefile: make the rst2html converter more strict
This will make the automated builds flag any problems.
Iustin Pop [Mon, 30 Aug 2010 09:10:06 +0000 (11:10 +0200)]
Add some more debugging functions
These are just variations of the standard debug, but are provided for
simpler code, since lazyness is something causing non-computation of
debug statements.
Iustin Pop [Thu, 26 Aug 2010 11:55:47 +0000 (13:55 +0200)]
Fix ReplaceSecondary moves for offline nodes
The addition of a new secondary on a node is doing two memory tests:
- in strict mode, reject if we get into N+1 failure
- reject if the new instance memory is greater than the free memory (not
available memory) on the node
The last check is designed to ensure that, irrespective of the other
secondary instances on this node, we are able to failover/migrate the
newly-added instance.
However, we should allow this, if the instances comes from an offline
node, which doesn't offer anything (not even disk replication).
Therefore this patch makes this check conditional on the strict mode.
Iustin Pop [Thu, 26 Aug 2010 11:55:16 +0000 (13:55 +0200)]
Update NEWS file
Iustin Pop [Thu, 26 Aug 2010 11:51:40 +0000 (13:51 +0200)]
Update man pages for the new -S option
Iustin Pop [Thu, 26 Aug 2010 11:28:59 +0000 (13:28 +0200)]
hspace: mark new instances as running
Otherwise the saved cluster state and the in-memory one are wrong.
Iustin Pop [Thu, 26 Aug 2010 11:07:40 +0000 (13:07 +0200)]
Implement cluster state saving in hspace
This also uncovered a few issues with the allocation model (instances
not being marked up, etc.).
Compared to hbal, hspace will generate either one or two files (for both
the standard and the tiered allocation mode), depending on the input
parameters.
Iustin Pop [Thu, 26 Aug 2010 10:58:38 +0000 (12:58 +0200)]
Change iterateAlloc to return the instance list
The Cluster.iterateAlloc and tieredAlloc functions are changed to also
return the updated instance list, since it is needed to have a “full”
cluster view.
Iustin Pop [Thu, 26 Aug 2010 09:49:19 +0000 (11:49 +0200)]
Implement cluster state saving in hbal
Also move the LUXI execution (-X) to the end, after all the output
messages are printed. No good in waiting for the messages for a long
while, especially as they are not up-to-date stats after the job
execution, just an estimation of what the state will be.
Iustin Pop [Wed, 25 Aug 2010 16:47:22 +0000 (18:47 +0200)]
Abstract the cluster serialization from hscan.hs
This is currently hardcoded in an internal function in hscan.hs, and we
move it to Text.hs for later use.
Iustin Pop [Wed, 25 Aug 2010 16:40:20 +0000 (18:40 +0200)]
Add a new option --save-cluster
This option will in the future be used to serialize the cluster state in
hbal and hspace after the rebalance/allocation steps.
Iustin Pop [Wed, 25 Aug 2010 16:04:41 +0000 (18:04 +0200)]
Add unittest for Node text serialization
This checks that the Node text serialization and deserialization
operations are idempotent when combined other.
Iustin Pop [Wed, 25 Aug 2010 15:53:53 +0000 (17:53 +0200)]
Switch unittest to custom hostnames
Currently, the hostnames are almost fully arbitrary chars, which breaks
the assumption that nodes/instances will be normal DNS hostnames.
This patch adds some custom generators for these hostnames, that will
allow better testing of text loader serialization/deserialization.
Iustin Pop [Tue, 24 Aug 2010 16:30:05 +0000 (18:30 +0200)]
Move text serialization functions to Text.hs
Currently these are in hscan, and cannot be reused easily.
Iustin Pop [Thu, 29 Jul 2010 04:03:42 +0000 (00:03 -0400)]
Fix a couple of typos in the manpages
Again, thanks to lintian.
Iustin Pop [Tue, 27 Jul 2010 18:44:30 +0000 (14:44 -0400)]
hail: fix error message for failed multi-evac
Currently we show the instance index, but this makes no sense outside
the current running program. Instead, we show the instance name.
Iustin Pop [Mon, 26 Jul 2010 23:49:23 +0000 (19:49 -0400)]
Update NEWS file for the 0.2.6 release
Iustin Pop [Tue, 27 Jul 2010 00:02:29 +0000 (20:02 -0400)]
NEWS: Add double blank lines before headers
This looks better for text-only viewing…
Iustin Pop [Fri, 23 Jul 2010 00:50:49 +0000 (20:50 -0400)]
hscan: return exit code 2 for RAPI failures
If some clusters failed during RAPI collection, exit with exit code 2 so
that tests can detect this failure.
Iustin Pop [Fri, 23 Jul 2010 00:32:41 +0000 (20:32 -0400)]
More enhancements to live-test.sh
Iustin Pop [Thu, 22 Jul 2010 13:57:13 +0000 (09:57 -0400)]
Fix another haddock issue
Iustin Pop [Thu, 22 Jul 2010 03:03:28 +0000 (23:03 -0400)]
Remove an obsolete function and add Utils tests
Iustin Pop [Thu, 22 Jul 2010 00:27:09 +0000 (20:27 -0400)]
Extend the live-test
The (recently-enabled) live test coverage stats found a few low-hanging
fruits in the tests we do…
Iustin Pop [Wed, 21 Jul 2010 23:25:44 +0000 (19:25 -0400)]
Use --union for hpc sum
… which fixes the issue noted in the previous commit (almost a brown
paper bag change).