rapi: Add new user option for querying
This was requested in issue 301. Before this patch, requests to“/2/query/*” and “/2/instances/*/console” would require authenticationwith a user with write access. Since that is not strictly necessary, anew user option named “read” is added....
Add Utility functions for loading data from test files
They mimic their python counterparts.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Improved TestHelper module docstring
Move gnt_cluster.SHOW_MACHINE_OPT to cli
This allows the option to be re-used in other places.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add ganeti-node-role ocf example file
This allows offlining nodes that don't respond if they are part of alinux-HA cluster.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add ganeti-master-role.ocf example file
This allows controlling the cluster master role if the nodes are part ofa linux-HA cluster.
Cleanup THH function use from built module namespace
Currently, THH.hs "injects" into the built code names of libraryfunctions like Text.JSON.makeObj, Ganeti.JSON.fromObj, etc. builtdirectly from strings, via (e.g.)
varE (mkName "makeObj")
This means that the "makeObj" name must exist in the target module,...
Switch Attoparsec/unicode test from QC to HUnit
This is not a quickcheck property, since it doesn't have anyvariable/arbitrary inputs. So let's make it a test case, and shorten abit the name.
Also, sorry for not catching this in the review.
Signed-off-by: Iustin Pop <iustin@google.com>...
Warn on invalid lines in HTTP user files
Without this change, invalid lines or values would be silently ignored.
Don't check for remote command directory as file storage
This test does not work properly if localstatedir is not “/etc”.
Document master daemon's shutdown behaviour
This wasn't clearly documented until now.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
pathutils: Add directory for remote commands
Also add tests to ensure it's never allowed as a file storage path. Aconstant for the lock file is also added.
utils.process.RunResult: Always set "fail_reason" attribute
Add previously missing node daemon GID to getent mock
The UID is there, the GID wasn't.
Add test utility to count calls to function
In some cases it's nice to verify a function has been called exactly Ntimes. This is going to be used in tests for remote commands.
Sort content of generated RPC code
This is similar to commit f5ce761. A stable sort order makes changesbetween versions easier to find.
Add new test for RAPI
Unlike existing tests, this actually tests RAPI at the interface withthe HTTP server. This way authentification can also be tested. A testfor “/2/query/…” is included as it's a bit special.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Add “gnt-job change-priority” sub-command
This can be used to change the priority of a pending or running job (thelatter only if there are unprocessed opcodes).
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Expose changing job priority via LUXI
A new LUXI request is added, in both Python and Haskell.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
jqueue: Allow changing of job priority
This is due to a feature request. Sometimes one wants to change thepriority of a job after it has been submitted, e.g. after submitting animportant job only to later notice many other pending jobs which will beprocessed first. Priority changes only take effect at the next lock...
jqueue: Set task ID for jobs added to workerpool
The job ID is re-used as the task ID, as job IDs are unique.
workerpool: Preserve task number when deferring
When a task is deferred it should receive the same task ID upon beingreturned to the pool.
workerpool: Add method to change task's priority
Using the task ID a pending task's priority can be changed. This will beused to change the priority of jobs in the workerpool.
workerpool: Change data structure for priority change
To prepare for the addition of a new function allowing changing apending task's priority, the internal data structure is slightlychanged. The (optional) task ID is stored as part of the task entry. A...
Automatically enable version-dependent GHC flags
Some GHC flags are very useful, but only appear in more recent GHCversions. To support the use of such flags while still supportingolder compilers, let's add conditional checks and enabling based onthe results....
Added attoparsec unit test for Unicode parsing
Attoparsec is known to have had issues with parsing non-ASCII strings.This test makes sure that parsing of Unicode characters works fine.
Signed-off-by: Michele Tartara <mtartara@google.com>[iustin: small doc string fixes]...
Add "Attoparsec" to the optional haskell packages
This will be needed for the data collectors of the monitoring agent.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Documentation for the NODE_RES level
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Improve make regen-vcs-version
"make regen-vcs-version" is run at every ./devel/upload time, in orderto have reasonably up-to-date information in the uploadedscripts/binaries. However, this means it currently triggersrecompilation of (at least) Ganeti/Version.hs, and sometimes more than...
Fix two hlint warnings
Sorry, I broke lint again :), by introducing two sub-standard changes.
Additionally, this silences an older existing warning that onlytriggers with some versions of hlint (e.g. 1.8.28 which is present inWheezy).
RunCmd: Expose "postfork" callback
The “_postfork_fn” parameter was only used for tests until now. Toimplement a good locking scheme, remote commands must also make use ofthis callback to release a lock when the command was successfullystarted (but did not yet finish)....
Merge branch 'devel-2.6' into master
Conflicts (both trivial): htools/Ganeti/Daemon.hs (_writePidFile rename)...
Merge branch 'stable-2.6' into devel-2.6
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Improve error message when migration status fail
Commit 6a1434d7 (“Make migration RPC non-blocking”) changed the APIfor reporting migration status, but has a small cosmetic bug: if themigration status if failure, but the RPC itself to get the statusdidn't fail, it shows the following error message:...
Fix type error in kvm/GetMigrationStatus
Commit 6a1434d7 (“Make migration RPC non-blocking”) changed fromraising HypervisorErrors to returning MigrationStatusobjects. However, these objects don't have an "info" attribute, sothey can't pass a reason back (which is in itself a bug); but the KVM...
Fix PID file writing in Haskell daemons
Currently, the code uses createFile, which has the effect of alwaystruncating the file. This is bad, as the content of the PID file iswiped even when we wouldn't be able to lock it!
We switch to openFd (createFile is just a wrapper over that), and we...
Remove unsafePerformIO usage
We need to change a few things, most importantly CLI options defaults,but otherwise we already used the path to files from functions whichwere already in the I/O monad, so we don't have to change much of thecode flow.
Additionally, Path.hs now has an explicit export list, to stop...
Change keyToFilename/readSSconfFile interaction
Currently, keyToFilename uses itself the default config path. In thelight of making that an function in the IO monad, let's remove the"default" path functionality from this function and make it alwaysrequire the config path; its caller, readSSConfFile, can then do the...
Change type of program options to 'IO [Options]'
Some options have defaults that depend on the environment, and wecould handle these in two ways:
- use a place-holder value (e.g. data X a = Default | Custom a) that is later read from the environment- move the options list to IO monad, where it can read the...
Two small logging improvements
The "starting" message is changed to match the Python one, and in casethe preparation fails, we also log the error (beside printing it onstderr or writing it to the error reporting pipe), as at this timelogging is usually set up....
Annotate confd startup/luxi binding error
The luxi binding is now annotated for better readability.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Switch writing of PID file to new reporting style
Currently, the writing of the PID file uses a "standard" errorreporting: catch exception, transform it into a 'Result' type, leavehandling of that to the caller. However, for daemon startup, weactually want exceptions to be propagated up until the handler which...
Move and rename BasicTypes.annotateIOError
It turns out that annotateIOError already exists in the standardlibrary, with a different purpose (this made me waste 10 minutestrying to understand why the types were wrong…). Let's move this outof BasicTypes, since it's used only in Daemon.hs, and also rename it....
Switch daemon startup to pipe-based error reporting
This changes the daemon reporting error to the same mechanism as inPython: use a pipe which is written to from the forked children.
Move preparation steps of confd in prepMain
This does the address parsing earlier, before we fork, forbetter/faster error reporting.
Split queryd run into prepare and exec
This will help with the general daemon split of prepare/run, and flagerrors earlier in the startup.
Convert tag objects to a safer type
Currently, we keep information about the "target" of a tag operationin a data type similar to (TagKind, Maybe String). This is unsafe, asnothing (at the type level) prevents us from accidentally having(TagCluster, Just "instance1.example.com"), or (TagInstance, Nothing)....
Switch Luxi TH code from simple to custom fields
This is needed so that we have more flexibility in generating Luxiserialisation code (deserialisation is still custom). Also, onlyexceptions are now using the 'simple' field types, so we might be ablelater to convert and remove that TH code as well....
Fix display of results in ==? operator
Doh, this is exactly the opposite of what we wanted… good thing notest failed so far :)
Make THH:genSaveOpCode a bit more general
This can be improved, by taking all hardcoded names as parameters, toserve as a more-general "build save clause for a multi-constructordata type". I'm not renaming the function as well, since I don't knowexactly how much we can abstract later....
sphinx_ext: Allow use of “rapi” module in pyeval
This way constants like “rapi.RAPI_ACCESS_WRITE” can be used indocumentation.
rlib2: Document two previously undocumented functions
Commit 208a6cff just included empty docstrings.
jqueue/mcpu: Determine priority using callback
Instead of being given the priority for acquiring locks by means of aparameter, mcpu will now call back. This is in preparation forimplementing a command to change a job's priority on the fly and allowsto change it while locks are being acquired (taking effect on the next...
Merge branch 'devel-2.6'
http/__init__.py: Remove extraneous argument
pylint complained, I fixed it, and unfortunately pushed too early.
rapi.testutils: Add utility to format HTTP headers
Once again this will be used by forthcoming RAPI test.
rapi.testutils: Return headers from mock utility
A newly added test for RAPI will also verify the returned headers. Atest in ganeti.rapi.client_unittest.py is split into smaller stand-alonetests.
http: Add wrapper for mimetools.Message
A newly added piece of code will also have to parse headers, so havingthis wrapper saves us from copying this part of code.
Rename Confd.hs to Confd/Types.hs
This should be the last module rename, promise!
We rename this to conform to the other hierarchies (e.g. Query), andto not have both Confd.hs and Confd/*.hs.
OpCodes.hs: add OpTagsSet and OpTagsDel
Also, adjust comment to $(genOpCode) block to avoid repetition of"only".
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
htools: move TagObject from Luxi.hs to OpCodes.hs
In addition to ReqQueryTags in Luxi.hs, the TagObject ADT is alsorequired for the "kind" attribute of OpTagsSet and OpTagsDel, whichare coming to OpCodes.hs next. Hence, we move TagObject there, andadjust imports accordingly....
Add missing tests for commit f0d2286
Commit f0d2286 changed the logic ofgnt_instance._ConvertNicDiskModifications to also allow a parameternamed “modify”. Unfortunately the corresponding unittest was notupdated. An “if”/“else” condition is also merged....
Implement base checkFn/prepFn/execFn model for daemons
This mirrors the code in the Python code base, and is required forclean error reporting during startup.
This patch implements the basic infrastructure; the confd daemon isnot yet modified to take advantage of this, just the types are...
One more ghc 7.6 fix
This is only in master, so needed to be fixed separately.
workerpool: Use itertools.count instead of manual counting
Instead of having to explicitely increment the value (“… += 1”), a callto next() is enough. These numbers should in no case be re-used (theyare used for ordering tasks). Using “itertools.count” is useful here as...
Merge branch 'devel-2.6' into submit
Conflicts: Makefile.am (reordering, fixed) htools/Ganeti/Confd/Server.hs (hlint fixes on master) htools/Ganeti/Daemon.hs (hlint)...
Switch devel/upload to a static file
We had twice in the past days questions about devel/upload being"broken", since bash re-reads shell scripts during their run and thisfile can get regenerated due to Makefile changes.
Since we only need this to be dynamically built for 3 variables, let's...
Fix compatibility with newer Haskell libraries
This small patch fixes compatibility with a few newer Haskell libraries:
- base 4.6, included with ghc 7.6, removed the deprecated 'catch' function from Prelude, so our "import Prelude hiding (catch)" is now...
Use SSH_LOGIN_USER rather than root for xl ssh
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix gnt-instance console with xl
- Rename xm-console-wrapper to xen-console-wrapper- Pass the xen command to use as a parameter
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add utility to check if file is executable
This replaces direct calls to “os.access” and“os.path.exists”/“os.path.isfile”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix NameError in constants.py introduced in merge 46c1f82
Add test for Luxi calls consistency hs/py
This tests that the same Luxi calls are defined in Python andHaskell. It doesn't test yet that their serialisation is correctthough.
Conflicts: lib/hypervisor/hv_xen.py: trivial
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>...
Disable E1101 on ganeti/http/server.py:424
Fix live migration under xl
Until now the only way to make live migration work in conjunction with"xl" was to add ssh known_hosts keys for every node's secondary ip onevery other node.
With this command we remove the target key verification: this is not...
Don't check for xend port when using xl
If the toolstack is set to "xl" we shouldn't ping xend for livenessbefore attempting a live migration.
utils.io: Improve handling of double and single slashes
Up until now “IsBelowDir("/", …)” would never return True. The reasonwas that an additional slash was added to the root path resulting in“//", which is “implementation-defined” in posix and treated specially...
workerpool: Don't mask variable in AddManyTasks
The name “priority” is already used.
workerpool: Simplify _WaitForTaskUnlocked
The function in is simplified in its structure and duplicated checkshave been merged.
cli.py: use None as name for tag operations on the cluster
This change is mostly cosmetic. Previously, the literal "cluster" wasused for the 'name' field of tag operations on the cluster (as opposedto a node or an instance). Since this field has a type of TMaybeString...
Fix previous merge
A call to _CalculateGroupIPolicy wasn't refactored during the merge.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix typo in gnt-instance man page
jqueue: Return jobs to queue when shutting down
When a job is still waiting for locks and the queue is shutting down,they should be returned and not actually start processing. Until nowjobs which transitioned from “queued” to “waiting” were alreadyconsidered to be running as far as the shutdown code was concerned....
gnt-debug delay: Add "--submit" option
Remove duplicate workerpool test
Commit 52c47e4e (July 2010) added the exact test twice, probably due toa copy & paste error.
Make hostname checks uniform between instance rename and add
Currently, we have instance rename doing extra checks on the hostname, to prevent accidental wrong renames; however, instance createdoesn't do these checks (issue 291), which (if DNS is misconfigured)...
Improve logging of new job submissions
This addresses issue 290: when receiving new jobs, logging isincomplete, and we don't have the job ID and/or summarieslogged. Only later, when the job is queried for or being processed, weknow more.
This is not good when troubleshooting, so let's improve the initial...
Improve handling of lock exceptions
There are two issues with lock exceptions right now:
- first, we don't log the original error; this is fine for now (locking.py always returns the same error here), but in general is brittle: if locking.py would start returning more information, we'd...
Add note about developing on a production machine
This is the bit of documentation missing for issue 170. Doingdevelopment on a machine which already has Ganeti installed kind ofworks, but only when the installed and the developed version are verysimilar, and even then it can be problematic....
Fix runtime memory increases
Commit 2c0af7da which added the runtime memory changes functionalityhad a small typo (wrong name); I've rewritten this to only compute thedelta once, for simplicity.
Fix validation of vgname in OpClusterSetParams
This variable can be empty, when we want to disable LVM, so we can'tuse TMaybeString.
Fixes issue 285.
Fix removal of storage directory on shared file storage
This patch makes _RemoveDisks symmetric to _CreateDisks with respectto file-based storage: _CreateDisks uses "in constants.DTS_FILEBASED",whereas _RemoveDisks was not update and only uses "==constants.DT_FILE". This results in stale directories left on the...
Switch non-redundant check to disk template-based
Currently, the warning/notice about non-redundant instances in clusterverify is based non empty secondaries list (how old is this?); theproper way to check this nowadays is via DTS_MIRRORED.
Fix permission for socket directory
The directory must we writable also by the confd daemon user.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>