Add local compression to OpBackupExport
OpBackupExport is extended by a compress parameter. This parameter(either 'none' or 'gzip') controls if instance disks are compressedbefore being sent over the network to the destination node.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Add --compress option to gnt-instance move
Add a new command line parameter to gnt-instance and pass its value toOpInstanceMove. Also document the option in the man page.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add possibility to compress to OpInstanceMove
OpInstanceMove now supports the 'compress' option. It allows to enablecompression for intra-cluster instance moves.
Remove trailing whitespace
Remove trailing whitespace from OpCodes.hs and OpParams.hs.
Remove blockdev_export RPC
This RPC call is no longer used, as for intra-cluster instance moves theimport/export daemon is used too.
Optimize dd parameters for instances moves
Benchmarks have shown that increasing the buffer size and disablingsynchronous output increased the throughput by about 10MiB/s, so makethose parameters the default.
Note that not using sync for output doesn't increase the risk of data...
Use import-export daemon for intra-cluster moves
This unifies the inter- and intra-cluster moves and the backup code, soless code needs to be maintained.
Also fix failing tests for LUInstanceMove by following the code changesin the tests.
Use cluster variable consistently
The cluster object is already retrieved from the configuration, so useit consistently in LUInstanceMove.CheckPrereq().
Correct comments
Correct comments which confuse the target node with the secondary nodeof an instance in LUInstanceMove.
Fix expectation of the htools wrapping
With the addition of hsqueeze, htools obtained a new personality.Additionally, the longer htool name also changes the line wrapping ofthe descriptions. Adapt the expectation accordingly.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Add a man page for hsqueeze
Describe the current state of the Ganeti dynamic power managementtool, hsqueeze, in a man page.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Add test for hsqueeze compression
In this example cluster, there are 6 nodes withcapacity for 4 instances each, holding a totalof 8 instances. So even after turning off 2 nodesand rebalancing, every node still has capacity for2 instances.
Add hsqueeze planning for compression
Add a new htool, hsqueeze, for dynamic power management. Thiscommit only implements the first useful part: plan (but notexecuting) taking nodes offline nodes still keeping within theresource limit.
Provide --target-resources option
Add a new option, to be used by hsqueeze, to specify thetarget free resources on each node. It is given as a multipleof the standard allocation, as specified in the instance policy.
Add RpcResult.__repr__ for debugging
This way it's possible to easily log the whole result in a humanreadable form.
Signed-off-by: Petr Pudlak <pudlak@google.com>Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Let git ignore `.hsenv` directory and vim's session file
If a configuration file can't be loaded, show which one
So far the error message only says it can't be loaded, but not what fileit is.
Refactor `getAllConsoleParams` to be more readable
Instead of creating a lot of lists and then zipping them, just workon every item one at a time. It makes the code much more readable.This changes the ordering of "effects", but the "effects" here are just...
Add dependency between test/py/docs_unittest.py and scripts/
test/py/docs_unittest.py tests that the scripts/ folder has beenpopulated and fails otherwise. The following sequence of commands hasbeen failing for this reason since the python bootstrap process changes...
QA: run all the instances during upgrade
The parameter 'upgrade-instances' was meant that one instance ofeach of these templates is left alive over the upgrade procedure.Threfore, breaking after the first suitable template is found isnot helpful.
Merge branch 'stable-2.10' into master
Merge branch 'stable-2.9' into stable-2.10
Merge branch 'origin/stable-2.10' into 'master'
Create check-news Makefile target
Checking the correctness of the NEWS file syntax is an important part of thecheck process, but up to now it was only possible as part of a bigger set oftests. This commit creates a Makefile target to run that independently....
Fix lint on qa_cluster.py
pep8 wants two blank lines after TestUpgrade. The previous commit gaveit one. This commit gives it the second.
Signed-off-by: Santi Raffa <rsanti@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add dynamic hbal entry to the NEWS file
This feature is new in Ganeti 2.10.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Update NEWS file about cpuid and soundhw
Add entry to the news file, per request.
Fix end-of-line whitespace
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Mention cmdlib unit tests and python-mock in NEWS
Add a NEWS entry about the cmdlib test framework and increased testcoverage, and mention that at least python-mock 1.0.1 is required as ofGaneti 2.10.
Run the cluster upgrade test in qa, if enabled
The test is not run directly after the cluster related tests,but only after basic istance creation is verified, as this testalso creates instances.
Add QA routine to test 'gnt-cluster upgrade'
The subtle points of upgrading a cluster can only be found inQA, so add a method to do so. A prerequisite is, that bothversions are installed on the test cluster. To have a realisticscenario, instances are created on both versions....
QA: Factor out the list of available instance test
The knowledge about which instance creating tests areavailable is best kept in the module on istances. Moreover,it is not only used by RunInstanceTests, but willalso be needed by the upgrade test.
Mention the --enable-symlinks option in INSTALL
From version 2.11 onwards, the default for Ganeti is to install asa standby version to be activated by the old Ganeti version itself.While this is useful for existing users, first-time installationsneed to add these symlinks or configure with ``--enable-symlinks``....
Mention gnt-cluster upgrade in UPGRADE
Ganeti 2.11 is the first version of Ganeti reachable (from 2.10) withgnt-cluster upgrade. Hence suggest this method in the UPGRADE notes.
Start NEWS entry for 2.9.0 rc4
So far, we have correct start/stop of luxid during gnt-cluster master-failoverinherited form stable-2.8
Update NEWS file about Hs2Py opcodes and constants
Add information about Haskell to Python opcode and constantgeneration.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Merge branch 'stable-2.8' into stable-2.9
Fix 'times' return type in the fake hypervisor
This patch changes the type of the times value returned by the fakehypervisor from string to int. The values are used during two RPCcalls, namely InstanceInfo and AllInstancesInfo, and caused errorswhen the more strict Haskell query code attempted to parse the...
Version bump for 2.8.1
Also, update the NEWS file accordingly.
Fix unit test regarding recent changes in node_info RPC
This fixes a unit test in instance_storage_unittest.py,which was affected by the recent change in the RCP'node_info'.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrovje Ribicic <riba@google.com>
Change default for --enable-symlinks to false
While in Ganeti 2.10, which is the first version to useversion-dependent symlinks in the sysconfdir should bringthese links by default, for later versions, these linksshould be created by 'gnt-cluster upgrade'. So change the...
Update NEWS file about move-instance improvements
Add information about disk template changes and using defaultiallocators on the target cluster.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Fix 'hvVncTls' key in 'hvsParameterTypes'
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
RPC: request spindle information properly
This patch fixes a problem with the RPC call 'node_info'.Depending on the exclusive storage flag, we need to adda storage unit for physical volumes in order to obtainstorage space information for them.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Remove deprecated deprecation warning for --no-lvm-storage
This removes the deprecation warning for the option'--no-lvm-storage'. The deprecation warning was intendedto be raised in 2.9 and 2.10 and removed after that.
Turn on the usage of Haskell instance queries
This commmit finally switches the instance query handling on.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Optimize instance query speed
The instance queries have a field called console which requires anextra RPC call. This commit allows extra parameters to be passed onto the query logic, which allows us to determine when to make theextra call.
Signed-off-by: Hrvoje Ribicic <riba@google.com>...
Tests for InstanceInfo retrieval added
As the retrieval of instance information is a tricky operation whichaffects the way the instance status is calculated, a few tests havebeen added to ensure its behaviour remains constant.
Add the console field
This patch adds the final console field, using the RPC calls of theprevious commit.
As this requires another RPC call, the live data retrieval functionsof the instance queries are modified and refactored slightly.
Haskell RPCs can now differ in parameters
The Haskell interface allowed multi-node RPCs to be executed inparallel, but with the same parameters every time. This restrictionis now lifted by providing a function call accepting pairs of nodesand parameters....
Add a console information RPC call
As the instance queries need console information, and the informationis retrieved through python classes that should not be ported yet, anRPC call supplying the information has been added. Some tests as well.
Add the last leftover fields
A few fields were missing from earlier work. The timestamp fields,the pnode.group.uuid field, and the custom nic parameters.Just the console remains.
Add aliases of existing instance fields
For legacy reasons, some instance fields are aliased. The aliases havethe same definition but appear under different names. Support for suchfields has been added to Query/Common.hsAs aliases can cause somewhat harder to find bugs, a new test was...
Rename Fillable fields to Indexed fields
The naming used in the previous commits did not reflect the use ofthe fields well, and clashed with the concept of fillable parameters.Thus some renaming was done, with slight documentation improvements.
Add instance network fields
This commit adds the instance network fields. Some of these fields arealso present for every NIC, requiring the reuse of functions from theprevious disk instance fields commit.
Add instance disk fields
The instance disk fields have been added. As these are the firstfields to be duplicated, some helper functions are present to supporteasier specification of fields which have to be duplicated for everyentry in an array of configuration objects, and numbered accordingly....
Add instance parameter fields
This patch adds instance parameter fields to the Haskell instancequeries. The fields retrieve maps of parameters related to e.g.hypervisors, or operating systems.Every instance can have parameters different from the defaults,...
Add further fields to instance queries
This commit adds some more fields that are simple to retrieve, asthey are properties of instances recorded in the configuration.
Add the first instance query live fields
This patch adds live fields which can be implemented with the existingRPC calls. Some fields require additional call types and will be addedlater. Most of them can be added at this point.The status field is especially interesting, as it depends not on the...
Add a new field type - FieldConfigRuntime
In the instance queries, some fields require both config and runtimeinformation. This patch adds a new field type for this purpose.
Move hypervisor fetches from Node.hs to Common.hs
As patches that follow will need the functionality inGaneti.Query.Node, this patch moves it to the common query file.
Enable 'AllInstancesInfo' RPC call in Haskell
The AllInstancesInfo call appears to have been implemented but neverused. The commit makes it usable by modifying the parameter type andserialization.
Unify all live calls which return no data
With the (current) instance query, there are three queries which do notrequire live data: instance, network, and group queries.The collectors and runtime types for these have been unified.
Initial changes for adding instance queries
Add basic instance querying capabilities to the Haskell codebase.Several fields are present, more to come.
Document the generic query function
The way the generic query works was not documented anywhere. Forgreater readability, add a short explanation to the genericQueryfunction.
Fix the "times" field type for KVM hypervisor
The KVM hypervisor Python class returned a "0" value for the timesfield, which gets serialized as a string. As Haskell is more pickyabout types, this caused the entire response to be declared as invalidand broke status reporting. Fixed by changing the value to a proper...
Verify that ConfD is running after master-failover
This patch adds additional 'gnt-node list' commands tothe testing of 'gnt-cluster master-failover' in order totest if ConfD (or LuxiD) is still running after amaster-failover.
daemon-util: handle luxid in {start,stop}_master()
Luxid was not handled in start_master() and stop_master() at all. As a result,during a master-failover, luxid would be left running on the old master andwould not start on the new master, leaving the cluster without management until...
Fix indentation of devel/build_chroot
This commit is separated from the previous one for reviewingconvenience. If git-blame points to this revision as the culprit ofyour problem, you probably have to go deeper.
Signed-off-by: Santi Raffa <rsanti@google.com>...
Support wheezy schroots
Add support for wheezy in build_chroot.
Prepare master branch for 2.11 devel cycle
Bump versions in various places to 2.11 and remove the downgrade codefrom cfgupgrade. Also adapt cfgupgrade tests.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Move orphaned unit tests to cmdlib/cluster_unittest.py
Through some merge from 2.9 to master, some unit testsin the file ganeti.comdlib.common_unittest.py wereno longer referenced in the Makefile and as such notrun anymore. With the introduction of the cmdlib test...
Fix bug with epydoc of BlockDev.GetUserspaceAccessUri
Signed-off-by: Santi Raffa <rsanti@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Fix typo in storage.FileStorage docstring
Signed-off-by: Santi Raffa <rsanti@google.com>Reviewed-by: Jose Lopes <jabolopes@google.com>
Fix path for serial file
It is actually located inside the queue directory.
Add implemented design docs to hidden toc section
This is required in order to calm a doclint warning.
Prepare version numbers for 2.10 release cycle
- Bump version numbers to 2.10 in various files- Move implemented designs to design-2.10
Merge branch 'stable-2.9'
Add unit tests for LUBackupExport
This patch adds unit tests for LUBackupExport.
Add unit tests for LUBackupPrepare
This patch adds unit tests for LUBackupPrepare.
Add unit tests for LUBackupQuery
This patch adds unit tests for LUBackupQuery.
Use node names in LUBackupQuery
The result of LUBackupQuery has to be indexed by node name rather thanby node UUID, otherwise the automatically constructed filter won'treturn any results.
Update design doc design-storagetypes.rst
This patch updates the design doc with the currentstate of the code base.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Make 'gnt-node list-storage' use default storage type
Currently, when no explicit storage type is given, the'gnt-node list-storage' command defaults to file storagewhether or not file storage is enabled on the clusteror not. This patch fixes it by defaulting to the default...
Make 'gnt-cluster modify' respect the order of templates
This fixes a bug where the order of enabled disk templateswas not respected when manipulating it with 'gnt-clustermodify'.
Mention reporting behavior in man page of 'gnt-node info'
This patch extends the man page of 'gnt-node info' todocument the space reporting behavior with respect tothe enabled disk templates.
Streamlining handling of spindles and default templates
This rather lenghy patch comprises a couple of refactoringsto achieve the following goals:- 'gnt-node info' should only report space information about spindles, when exclusive storage is enabled on the node...
Rename entities of unit test for MakeLegacyNodeInfo
This is a small preparation step for the next patch.It just renames the key/value constants for the unittest of MakeLegacyNodeInfo, because they were ratherundescriptive and thus made changes to the test cumbersome....
Add test suite for some names in 'AutoConf'
Add test suite for some names in 'AutoConf' to test whether the valuesgenerated from 'configure' and the 'Makefile' are valid filepaths,user names, group names, among others. Fixes issue 601.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Fix several search path configure options
Fix several search path configure options to properly handleseparators.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Design document for hsqueeze
This design document describes a new htool, hsqueeze, tobe used to move the externally-mirrored instances in such away that as few nodes as possible host instances at all.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Martin Zobel-Helas <zobel@debian.org>...
Version bump for 2.9.0 rc3
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add NEWS entry for 2.9.0 rc3
The main news is, that we fixed another issue with the configurationupgrading process.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Remove incorrect comment
Removing a comment which falsely states that Ganeti willstop supporting certain upgrades at some point.
cfg auto update: match ipolicy with enabled disk templates
This patch fixes issue 599. The problem was that with 2.9,a sanity check was introduced that made sure that theipolicies' set of disk templates is a subset of thecluster-wide enabled disk templates. In order to make...
Remove obsolete configure option for shared file storage
This patch removes an obsolete configure option relatedto shared file storage. It was remove a while ago, butaccidentally reintroduced through a merge.
Fix typos and indentation for cluster.py
This patches fixes a typo and some indentation errorsthat were accidentally introduced by premature pushingof the patch series "Improvements on disk templates, inparticular file storage".
Improve harep documentation
Add a more complete description of how the tool works.
QA: remove subsumed QA test wrt disk templates
This patch removes a QA test that was supposed to testthat unused disk templates can be disabled safely. Thetest is now subsumed by unit tests and thus we remove ithere. Besides that, it was actually testing the wrong...