History | View | Annotate | Download (6.9 kB)
Change the Node.group attribute
Currently, the Node.group attribute is the UUID of the group, as untilrecently Ganeti didn't export the node group properties. Since it doesso now, we make the following changes (again apologies for a bigpatch):
- we change the group attribute to be an index, similar to the way an...
Text.hs: also save the group data when serialising
This should have been in the previous patches, but sent separate forclarity.
The live-test script is updated to read the first node from thecluster, now that the text files don't start anymore with the node...
Text.hs: change to use sepSplit
The new sepSplit function can split based on empty lines, so we removethe hackish text splitting from before and simply use sepSplit. Thisis needed as the addition of extra sections would have increased thecode linearly, which we don't want :)...
Text.hs: also read cluster tags from the data file
This means that a file with the correct information is as accurate asthe other backends (Luxi, Rapi). Serialization of tags is in the nextpatch.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
Text.hs: serialize cluster tags when writing data
This is the complement to the reading part. Now the live-test workscorrectly against clusters with configured exclusion tags.
Text: read/write the allocation policy
Rework the data loader pipelines to read groups
This (invasive) patch changes all the loader pipelines to read the nodegroups data from the cluster, via the various backends. It is invasiveas it needs coordinated changes across all the loaders.
Note that the new group data is not used, just returned....
Rework the types used during data loading
This improves on the previous change. Currently, the node and instancelists shipped around during data loading are (again) association lists.For instances it's not a big issue, but the node list is rewrittencontinuously while we assign instances to nodes, and that is very slow....
Convert some leftovers to NameAssoc
The type alias NameAssoc has been introduced a long time ago, but thereare some few not-yet-converted cases. In preparation for changes to thattype, let's make sure we use it consistently.
Signed-off-by: Iustin Pop <iustin@google.com>...
Add a type alias for UUIDs
This is to pottentially allow easier changes later.
Text: read/save the node group UUID
Compatibility with old text files is kept by using the default UUID ifthe file (or even some records) don't have a UUID.
Node: add the node group's UUID
This is not used anywhere yet, and the backend are all just adding thedefault UUID, not the real one.
The patch also allows displaying the group UUID in the node list.
Text loader: split parsing from loadData
This change, which will be followed by similar changes in the otherloaders, splits the parsing of the data from the actual loading fromdisk. Since the parsing doesn't usually involve IO actions, we will beable to better test the parsing. The loading becomes a smaller part of...
Abstract the cluster serialization from hscan.hs
This is currently hardcoded in an internal function in hscan.hs, and wemove it to Text.hs for later use.
Add unittest for Node text serialization
This checks that the Node text serialization and deserializationoperations are idempotent when combined other.
Move text serialization functions to Text.hs
Currently these are in hscan, and cannot be reused easily.
Add more unittests
Instance, Node and Text modules have improved coverage.
Switch the text file format to single-file
This patch changes from the two separate files to a single file, withsections separated by a blank line. Currently only the node and instancedata is accepted, later the cluster tags will be read too via thisformat....
Change the signatures of the text loader slightly
This is in preparation for the text format changes.
Introduce support for reading the cluster tags
While these are not actually populated from the backends, and all theprograms ignore them, this patch contains the changes in the functiontypes required.
Add a tags attribute to instances
… and read it in all the loaders. hscan is modified to save it to thefiles it generates.
The attribute is not yet used in any place.
Small change in some list arguments
This is simpler than the concat operator.
Move some utility functions to Utils.hs
These were already duplicate (Text and Simu) and we need tryRead in more places.
Add some more instance tests
This include instance text load tests.
Add a simulated cluster data loader
This is useful especially for hspace, where we might want to simulate ahypothetical cluster to check allocation beforehand.
Fix hlint-generated warnings
This big patch cleans up the code per hlint indications. Many removalsof extra parentheses, replacements of concat . map with concabtMap,extra dollar signs, eta reductions, etc. were performed.
The code still compiles and passes a couple of manual tests on sample...
Fix the various monomorphism warning
In a few places (e.g. tryRead or any printf call) it's a little bit hardto add the correct type signatures, but in the it is possible to fixthese warnings (which can bite one in subtle cases).
Add cpu-count-related attributes to nodes
This patch adds cpu-count related attributes to nodes: - total cpus - cpus in use - ratio of virtual:physical cpus
We also set correctly the cpu values at load time, but we don't doanything yet while moving instances around. The cpu ratio is shown in...
Add a new vcpus attribute to instances
This patch adds reading of vcpu count for instances, in preparation forusing the vcpu ratio in cluster scoring.
Add copyright/license information
This doc-patch adds copyright and license information to (hopefully) allneeded files.
Lots of documentation updates
This patch does only doc build changes, doc changes and function movearound (for more logical documentation). It should have no impact at allon the code.
Add type synonyms for the node/instance indices
This is a first step towards full datatype renaming. That requires morechanges, so at first we only want to document clearly what is a nodeindex, what is an instance index, and what is a plain Int.
Fix some haddock issues
Slash is a reserved char. Slash is a reserved char. Slash is a…
Remove ktn/kti from first half of loader
This patch removes the ktn/kti lists from most parts of the first halfof the loading sequence. Some remain as the [(String, Int)] is thenicest way to lookup names and get indices back.
Add a small class for Nodes and Instances
Since both nodes and instances support some common functionality (namesand indices), we add a class so that we can access these attributes in ageneric way.
Add back names to nodes/instances
In order to simplify the data structures, we add back the name on thenode and instance objects. We still keep the index for, well, indexing,but we will use the name directly from the object, in order to get ridof the ktn/kti arguments which are passed around everywhere.
More code reorganizations
This new big patch does a couple of more cleanups in the loading of datachapter: - introduce a Types module that holds most types (except the base Node/Instance/etc.) so that multiple other modules can use these (instead of only Cluster and its users)...
Rework the loader model
This big patch changes the loader model from “string data as commonformat” to actual object structures as common format.
The text loading function move from Cluster.hs to a new Text.hs module,some common functions are moved to a new Loader.hs module, and the...