Fix missing '@raise' in docstring
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Fix missing 'L{...}' in docstring
Rename 'CdromOption' to '_CdromOption'
... because it is a protected definition.
Fix whitespace
Merge branch 'stable-2.11' into master
Add andRestArguments to IDiskParams
In this way, we cann pass through the opaque parametersrequired for disk creation and modification in the case ofexternal storage.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Add function providing the canonical andRestArguments
The field catching the remaining fields will always be of the sameshape, so add a function for this to make usage simple.
Add genAndRestArguments :: Gen (Map String JSValue)
So that objects using AndRestArguments are available for testing.As the AndRestArguments are intended for passing through additionalparameters passed on the command line, we restrict them to thevalues that are allowed at this position: Strings of ASCII characters...
Add additional constructor AndRestArguments to OptionalType
A field of this type will capture all the remaining fieldsof an object as JSValues. Obviously, the intended use isto have precisely one such field. This mechanism will allowto pass opaque values trough, as it is, e.g., required for...
Fix specification of TIDiskParams
Commit 580b1fdd incorrectly assumes that disk parameters arejust the standard ones, whereas the man page explicitly statesthat additional parameters can be passed as well, if they makesense for the chosen storage type. Fix this....
Make BlockDev subclasses adhere the interface for Create
In commit 702c3270 two new parameters were added to theCreate function of BlockDev. Make subclasses also adherethis specification.
Make BlockDev subclasses adhere to new interface
In commit 702c3270 two new parameters were added to theconstructor of BlockDev. Make the subclassess accept theseadditional parameters as well.
Make disk.name and disk.uuid available in bdev
Until now Disk name and uuid was not available on bdev level.In case of ExtStorage, this info is useful, and may be for othertemplates in the future too.
This patch treats the name and uuid object slots just like the size...
Add script that checks the copyrights of a dirty local repo
Script to check whether the local dirty commits are changing fileswhich do not have an updated copyright.
The script will determine your current remote branch and local branch,from which it will extract the commits to analyze. Afterwards, for...
Add 'dnsmasq' section and TAP interface names
User new error handling functions in SubmitJobToDrainedQueue
This somewhat shortens and simplifies the code.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Use new error functions when querying locks
This helps to handle errors coming from the Luxi client.
Use new error handling functions for querying jobs
Since we already touched getJobIDs, and this function is already basedon ResultT, use new error functions here as well.
Update getDirJobIDs to use ResultT
Also simplify code and remove unused functions.
Improve liftIO of ResultT to catch IOErrors
Any IOErrors are now captured in ResultT's liftIO using try and converted into'Bad'. This prevents IO exceptions from leaking into ResultT code and allowsthem to be handled in a pure way inside ResultT's sum type....
Rename 'resultT' to 'toError'
.. to better correspond to its generalized type.
Generalize type signatures of functions that produce Result
This allows them to be used in any 'MonadError', in particular in monad stackscomposed using ResultT, without explicit lifting.
Add withErrorLogAt which annotates and logs errors
.. and rethrows them within a MonadError. This allows to just log errorsand let them be handled elsewhere.
Add functions for manipulating errors in Result(T)
There is often need to manipulate these errors, for example to convert a Stringfrom Result into an exception. These functions make this easier.
Function 'toErrorStr' lifts 'Result' to any 'MonadError'. This is useful...
Refactor instance MonadLog (ReaderT r m) to unify style
.. with MonadLog (ResultT e m) to be more point-free.
Add instance MonadLog for ResultT
This allows to log in monad stacks containing ResultT.
Remove FromString in favor of Error from standard libraries
They have the very same functionality, and using our own FromString onlycauses unnecessary code duplication.
Fix bracket style in rapi-workload
Pep8 complained about bracket indentation. This fixes it to satisfy Pep8as well as our style guide.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Replace duplicated code with readJSONWithDesc
JSON errors were annotated at other places, so use the function there aswell.
When parsing fields, annotate JSON errors with names
This allows better debugging of failed JSON conversions.
Add a function that enhances readJSON error messages
If an error occurs, the error message is annotated with a given textualdescription of the parsed type and optionally also with the input data.
Make safeRenameFile create dirs with defined permissions
If, and only if, safeRenameFile creates a new directory, make sureit has well defined permissions. While there, also optimize for thecommon case. The main use of safeRenameFile is archiving jobs. As...
Add constant for subdir permissions within the job queue
When archiving jobs, new directories have to be created, asjobs are archived in groups of 10000. Add a constant describingthe permissions of these newly created directories.
Note that, due to the type, the constant cannot be part...
Add utility to fix permissions
Especially when creating new directories, we need to make sureownership and permissions are set correctly. Provide a functionto do so.
Add data type describing permissions and possibly owners
When creating new files, and, more importantly, new directoriesit is relevant to set permissions, and possibly owners, correctly.Provide a type specifying the target configuration.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Add missing import
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Merge branch 'stable-2.10' into stable-2.11
Enable network tags in Haskell code
Prior to the creation of the 2.10 branch, network tags were broken, andthe Haskell code introduced there mistakenly accepted this as thedesired functionality. This patch fixes this in a very simple way.
Signed-off-by: Hrvoje Ribicic <riba@google.com>...
Merge branch 'stable-2.9' into stable-2.10
Add 'provider' to IDiskParams
IDISK_PROVIDER was included in python's IDISK_PARAMS, so itshould also be included in the Haskell code.
Now that luxid creates and enqueues jobs, without this patch theExtStorage interface is broken as the user can not pass the disk...
Haskell style: explicit errors, visible in the type
While it is already standard in the Ganeti code base, explicitlydocument that we do not like Exceptions and use sum types instead.
Clarify spacing in record syntax
So far, our code base does not have a consistent way of spacingrecords. To work towards more consistency, add a recommendationinto out style guide. We standardize on what seems most commonin the Haskell world and also is the dominant form in our code...
Improve the point-free section of the style guide
Distinguish declaring functions in the point-free style and usinga very similar technique to avoid parentheses (which isn't technicallypoint-free).
Add code style document to documentation
The Ganeti code style has been stored on the project wiki at:
https://code.google.com/p/ganeti/wiki/StyleGuide https://code.google.com/p/ganeti/wiki/HaskellStyleGuide
This commit combines the two pages into an .rst file with minimal...
Export docs from wiki to repo
Export documents about 'cluster keys replacement' and 'design filebased storage' from the wiki to the repository.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Revision bump for 2.9.3
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Jose Lopes <jabolopes@google.com>
Schedule 2.9.3 release
Fix chroot for sid to include the development tools
Disabling client certificate usage
This patch temporarily disables the usage of the clientSSL certificates. The handling of RPC connections had aconceptional flaw, because the certificates lack a propersignature. For this, Ganeti needs to implement a CA,...
Implement auto-archiving of jobs
As luxid is taking over the handling of the job queue, italso needs to handle the automated archiving of jobs. Herewe replicate the semantics of the current python implementationof archiving as many jobs older than the given time as possible,...
Add a utility function to try archiving jobs
Provide a function that walks through a list of job ids andarchives them if appropriate. Abort that process if a giventimeout is reached.
Support computation on Timestamp
As timestamps are also used to determine if an event is sufficientlylong in the past (e.g., on archiving jobs), support adding a timeinterval to a Timestamp.
Add constructor function for Timestamp
Provide means to get Ganeti's internal timestampsfrom standard clock time.
Add a predicate on Jobs on whether it can be archived
Jobs usually are archived a given time after they have finished.For finalized jobs without end-time, the start-time is taken enlieu. This function provides the pure predicate for this decision.
Make ArchiveJob in luxid create the archive, if necessary
As jobs are archived in groups of 10000, creating new subdirectoriesof the archive might be necessary when archiving a job. Use afunction that takes care of this.
Provide a safe version of rename
...that also creates the target directory, if needed.
Fix expectation for the return value of jobqueue_rename
On sucess, jobqueue_rename returns a list containing onenull per change request.
Document fix of issue 691 in NEWS
NEWS: fix typo in 2.8.4 release
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Petr Pudlák <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Fix 'hvparams' of '_InstanceStartupMemory' on hypervisors
Most hypervisors were calling '_InstanceStartupMemory' but not passingthe 'hvparams' keyword argument. Actually, it is not necessary topass this argument given that it is an attribute in the instance...
Add missing option to gnt-instance documentation
Signed-off-by: Hrvoje Ribicic <riba@google.com>Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Update NEWS file
With the merge of stable-2.8 into stable-2.9, quite a few fixesgot inherited.
Merge branch 'stable-2.8' into stable-2.9
Fix disk_type error in hypervisor parameter documentation
According to the code, presenting disks as paravirtual is supported onboth HVM and KVM, while IDE works only on KVM. This patch updates docsto be accurate.
fix off-by-one error in indentation
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Version bump for 2.8.4 and NEWS update
Update the version number to 2.8.4 and insert the final details for thisrelease in the NEWS file, including the release date.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Implement ArchiveJob queries in luxid
With luxid taking over the tasks of masterd, archivingjobs also belongs to its responsibilities. As archivinga job affects the global state of the job queue, synchroniseover the queue lock.
Add RPC call jobqueue_rename
Archiving jobs is also replicated to all master candidates.Therefore luxid needs to be aware of this RPC call.
Redesign communication mechanism in light of limitations
Redesign the communication mechanism in light of implementationlimitations that have recently come up in prototypes using KVM.
Anchor OS reinstall design doc in Makefile and index
Add OS reinstall design doc to the list of design docs in theMakefile, otherwise it does not get compiled when modified, and add italso to the index page of the documentation, where all the otherdesign docs are anchored....
Run drbdsetup syncer only on network attach
As late as DRBD 8.3.11, the drbdsetup syncer command has a bug causingnodes to hang from time to time, requiring manual intervention to fix.The use of the command cannot be avoided, but the incidence of use can...
Make use of fieldListToFieldMap
...to avoid duplicating that code all over Ganeti.Query.
Provide a utility function to map FieldList to FieldMap
As the same construction is used in several places, it is betterto have it factored out as a named function.
Add query support for locks to luxid
While requests only get forwarded, it still helps toget luxid feature-complete with respect to master.
Provide fields for lock queries
For luxid to be feature-complete with respect to masterd, italso needs to answer requests about locks. This includes knowingthe fields available for locks.
Update NEWS file with news about job cancellation bugfix
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Fix QA flakiness
The newly added job QA has some flakiness with respect to its use ofgnt-job watch. Fix this by waiting until the canceling status isreplaced with the canceled status, or a timeout is reached.
Linting fix: remove unused import
Remove dead Ganeti.Query.Job.loadRuntimeData
This function was exported from the module, but actually neverused anywhere in the code base. So clean it up.
Update NEWS file: issue 687 and configure fix
Add entries to the NEWS file for the two user-visible changes thathappened since the last update: issue 687 got fixed, and configurenow supports Sphinx versions 1.2+.
luxid: fix detection of master node in node query
Ganeti.Config.getNodeRole would rely on clusterMasterNode returning themaster node name, however clusterMasterNode returns the master node'sUUID. We fix this and a similar issue in Ganeti.Query.Node.nodeFields....
Add missing parameter entry to man file
The gnt-instance manual was lacking an entry for the vnc-password-filehypervisor parameter. This patch adds one, and also some information onthe default value of the parameter.
Add QA test for job cancellation
This patch introduces a QA test in which a job is cancelled whilewaiting.
Add correct locking of master node to gnt-debug delay
The gnt-debug delay command required locks for all nodes except themaster - this patch fixes the issue by adding master to the lockswhenever needed.
Add job id type assert to jqueue.py
While the changes introduced in previous patches should stop any jobid parameters reaching the queue as strings, add an assertion here tocatch any strings making it through.
Add job id transformation/check to Luxi Python client
This patch adds checks to the Luxi client, making sure that job idsare converted from strings to ints before being passed on, or that anerror is reported.
query: fix detection of master in _GetNodeRole()
Commit 1c3231aa changed the invocation of _GetNodeRole() to pass themaster node by UUID and not by name, but didn't change theimplementation to compare the nodes by name. As a result, the masternode (which is also a master candidate) would always fall through to the...
When updating job queue, support virtual paths
When replicating parts of the job queue, allow for virtualpaths in the RPC call. In this way, replication will alsowork correctly in a vcluster setup. Note that makeVirtualPathlives in IO, and hence cannot be part of the pure encoding...
Add a module to support virtual clusters
Virtual clusters are an efficient way to test how Ganeti behaveson a large cluster without requiring a large number of machines.Now that more tasks like job replication are done by luxid, providethat functionality in Haskell as well....
Move vcluster-related constants to Constants.hs
...as, in that way, they will also be available in Haskell,where job replication happens as well.
Start-master/stop-master always fail if confd is disabled
In 'daemons/daemon-util.in', 'start-master' and 'stop-master' alwaysfail if confd is disabled.
Fixes issue 685.
Signed-off-by: Jose A. Lopes <jabolopes@gmail.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Fix KVM cdrom image URL in the second cdrom drive
kvm: check that the ISO image is there if it's a URL
Perform a simple urllib2 check on ISO images specified as URL beforeinstance start, so as to work around qemu bug #597575 [1].
[1] https://bugs.launchpad.net/qemu/+bug/597575
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>...
Allow KVM to boot from HTTP
New versions of KVM support booting from HTTP-hosted ISO images, vialibcurl. This patch adds a proper check to allow defining either a sane,absolute path or an HTTP URL as an iso image path.
Remove "format=raw" from the cdrom device options when iso_image starts...
Include target node in hooks nodes for migration
In case of DRBD, hooks run on both primary (source) and secondary(target) nodes. To get the same behavior for DTS_EXT_MIRROR, where wedo not have secondary node, we should explicitly add target node tohooks nodes during instance migration/failover....
Fix configure dirs
Fix overridable configure directories, namely, --bindir, --sbindir,and --mandir.
Variables 'bindir', 'sbindir', and 'mandir' are exported from'configure.ac' to 'Makefile.am'. At first, it would seem to bepossible to do the following:...
Docs: Update the version of ganeti-instance-debootstrap
The installation manual used version 0.12, which had some bugs.This patch updates it to the latest 0.14.
Reset version-specific downgrades
As stable-2.11 has been branched off and revisions bumped, downgradesare now supposed to go to 2.11. Currently they are a no-op and, inparticular, ssl-certificates are still expected in 2.11.