Fixes to pass py-apidoc (make commit-check)
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Iustin Pop <iustin@google.com>
Fixes to pass pep8 (make lint)
Fixes to pass pylint (make lint)
Fixes to pass unittests (make check)
Check for existing networks/groups in Network LUs
Add tags in network objects
Support: - gnt-network add --tags. - gnt-network list-tags/add-tags/remove-tags/. - gnt-network list -o +tags
Change default fields for network listing
Pass detailed network info in hooks
Pass netinfo in rpcs
If a nic has a network field then encapsulate a network object innetinfo slot for every rpc. This is needed to pass network info toscripts managing nics (kvm-vif-bridge).
Introduce _BuildNetworkEnv().
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>...
cfgupgrade for networks
Add networks slot in config data.Add networks slot in nodegroups.
Use network's mac prefix
Modify existing GenerateMAC so that it takes network as an argument.
Substitute _GenerateOneMAC with a decorator _GenerateMACPrefix thatchooses the prefix (network's or cluster's) and wraps_GenerateMACSuffix that generates the three remaining bytes....
Include networks in Makefile and bash completion
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Iustin Pop <iustin@google.com>
Add gnt-network man page
Rapi support for networks
Support: - GetNetwork(s) - CreateNetwork - ConnectNetwork - DisconnectNetwork - RemoveNetwork
ss_conf support for networks
Introduce client support for networks
gnt-network is used to manipulate and handle networks that currentlyprovides the following operations:
Modify instance client to support networks
Add --no-conflicts-check option.Modify instance query methods to return nic's network info.
Peripheral changes for networks
Specifically: * export a new environment variable for the gnt-os-interface * export a new environment variable for KVM specific scripts * ovf support for networks
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>...
Modify LUInstanceSetParams() to support networks
Keep backwards compatibility just like LUInstanceCreate() and supportnetwork parameter for NIC object.
Add ec_id argument in Update() to be able to commit temporary IPs thatwere previously reserved.
Modify RemoveInstance() to support networks
Release any IPs held the instance before actually removing.
Modify LUInstanceCreate to support networks
Implement backend support, to export the IP pool managementfunctionality to the clients. When the new NIC parameter 'network' isgiven, the ippool management system is triggered. If a NIC belongs toa network, it inherits the netparams (mode, link) as its nicparams. If...
Config methods for reserving/releasing IPs
Use TemporaryReservationManager for IP handling to ensure config dataconsistency.
Implement: - methods for reserving/releasing IPs - _UnlockedCommitTemporaryIps() - CheckIPInNodeGroup() used for conflicting IPs...
Implement network/nodegroup mappings
LUNetworkConnect() connects a network to a nodegroup and definesits netparams (mode and link). Specifically: * Check if network already exists * Check if netparams are valid * Check if already connected to the same nodegroup...
Implement LUNetworkSetParams
Support modifying all network parameters except for network (iprange). Cannot modify gateway and reserved ips at the same time.
Implement LUNetworkQuery
Summarily list all existing networksSupply detailed info for every existing network - List used/free IPs - List instances with NICs assigned to the corresponding network - List NIC index and IP for the above instances
Implement complementary config methods for retrieving networks....
Basic IP pool management logic
Implement LUs for corresponding opcodes: * LUNetworkAdd: - Check for IP validity - Reserves all necessary IPs - Create new Network config object * LUNetworkRemove: - Checks if connected to any nodegroup - Remove a Network config object...
Introduce new module for IP pool management
Add new library module lib/network.py.Introduce new class: AddressPool.
AddressPool implements all operations needed for managing IPs insidethe IP pool.
Given a Network config object (nobj), the class:
IP pool related objects, opcodes and constants
Config objects: * Introduce L{Network} with - IPv4 network field (mandatory) - IPv4 gateway, IPv6 (network/gateway), mac prefix, type (optional) * Modify existing config objects to support networks: - Add new slot 'network' to L{NIC} config object...
Add more basic validation types
This mirrors the ht.py types PositiveInt, NonNegative, etc., exceptthat they work at a more generic level (any numeric type, respectivelyany non-empty list).
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Adeodato Simo <dato@google.com>
Fix a hlint warning
This is yet-another version-specific warning (fixed in later hlintversions), but it's better to keep it clean.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add DRBD parser unit tests
This adds tests that existing test files can be parsed by the Haskellparser as well, plus one new test file.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add parser for DRBD /proc file
Signed-off-by: Michele Tartara <mtartara@google.com>...
Properly log errors when setting up daemon FDs
While writing the pipe-based reporting and trying various ways tobreak the startup, I fought for a while trying to understand why errorreporting was different when running the daemon as a user (with norights). It turns out that setupDaemonFDs wants to open the log file...
Switch opcode data type from normal to record constructors
Currently, the OpCode definitions are using normal constructors:
data OpCode = OpTestDelay Double Bool [String] | OpInstanceFailover String Bool (Maybe String) …
While this works for a few opcodes, it becomes unwieldy when dealing...
Fix docstrings for the Filter type
While looking at the opcode docs and clicking accidentally on thefilter type, I saw that the haddock formatting is broken due tonon-escaped use of special chars. Let's convert the ascii-likeformatting to haddock, and have nicer apidoc....
Simplify a few test helpers
Use more monadic combinators instead of explicit code.
Add custom code for CV_E* constants to convert-constants
Currently, the cluster verify errors are defined as follows:
CV_ECLUSTER_FOO = (TCLUSTER, "ECLUSTER_FOO", "description")
This means there's no standalone name for the string "ECLUSTER_FOO",which makes it hard to derive automatically a type for this union in...
Improve error message for wrong opcode parameter values
When given an empty string as value to a parameter that doesn't acceptempty strings, the error message was confusing, since it was showingjust as a double space.
Signed-off-by: Iustin Pop <iustin@google.com>...
Cleanup ht's use of positive/strictpositive
Currently, ht.py uses a bad terminology for positive/non-negativenumbers. Per http://en.wikipedia.org/wiki/Positive_number, this is thecorrect terminology:
- A number is positive if it is greater than zero.- A number is negative if it is less than zero....
Split OpCode.hs and add module for opcode parameters
Due to TemplateHaskell stage restrictions, we can't define parametersin the same module as we're using them for TH, so we have to defineall module parameters in a separate module.
This patch therefore splits OpCodes.hs in two, adding that module and...
Create a new Ganeti/Types.hs module
There are already three cases where we copied type definitions betweenthe htools-specific types into the main ganeti code. Let's stop doingthis ☺ and create a common types module that holds these.
Note that there already exists BasicTypes.hs, but that refers to very...
locking: Simplify condition
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
configure: Shell cleanup
- Remote unnecessary quote characters- Change some uses of “test "$foo"” to “test n "$foo"” Merge “if/else/if” into “if/elif”- Wrap lines longer than 80 characters and add a test to “make check”
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Add opcode for running commands remotely
The opcode doesn't pay attention to the build-time flag to enable ordisable restricted commands. In a cluster different nodes could havedifferent settings.
Node locks are acquired in shared mode by default, but the use of an...
Add RPC for restricted commands
Expose the functionality to run restricted commands remotely via RPC.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
locking: Don't schedule pending acq. for short timeout
Scheduling a pending acquisition is relatively expensive and lot of codeis involved. Unless there is already one, a new pipe needs to be opened.Data structures need to be updated as well, only to be undone shortly...
Prevent hlint crash on unicode chars
Some versions of hlint crash on the Attoparsec test file because theyare not able to correctly handle unicode characters if the proper encoding(--utf8) is not specified.
Rename design document for SSH setup
More design details will be added regarding setting up the node daemon,so this is more about adding a node than just setting up SSH.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
backend: Implement remote commands
As per design document (doc/design-remote-commands.rst), a number ofrather strict tests is applied to any incoming request, a delay isinserted upon errors and returned error messages are very generic(unless it's the actual command that failed). There are unit tests for...
configure: Add option to enable remote commands
By default remote commands are disabled and need to be explicitelyenabled at build time.
Add unit test for RAPI handler access definitions
- Ensure query-related resources have the same access permissions (specifically “/2/query/*” and “/2/*/console”)- Check access permission consistency (write implies read)
Update ganeti-rapi man page for new user option
Instead of duplicating what is already described in the fulldocumentation, a reference is added instead. Man pages can't useconstants or assertions, therefore it is easier for them to get out ofsync with the code....
rapi: Add new user option for querying
This was requested in issue 301. Before this patch, requests to“/2/query/*” and “/2/instances/*/console” would require authenticationwith a user with write access. Since that is not strictly necessary, anew user option named “read” is added....
Add Utility functions for loading data from test files
They mimic their python counterparts.
Improved TestHelper module docstring
Move gnt_cluster.SHOW_MACHINE_OPT to cli
This allows the option to be re-used in other places.
Add ganeti-node-role ocf example file
This allows offlining nodes that don't respond if they are part of alinux-HA cluster.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add ganeti-master-role.ocf example file
This allows controlling the cluster master role if the nodes are part ofa linux-HA cluster.
Cleanup THH function use from built module namespace
Currently, THH.hs "injects" into the built code names of libraryfunctions like Text.JSON.makeObj, Ganeti.JSON.fromObj, etc. builtdirectly from strings, via (e.g.)
varE (mkName "makeObj")
This means that the "makeObj" name must exist in the target module,...
Switch Attoparsec/unicode test from QC to HUnit
This is not a quickcheck property, since it doesn't have anyvariable/arbitrary inputs. So let's make it a test case, and shorten abit the name.
Also, sorry for not catching this in the review.
Warn on invalid lines in HTTP user files
Without this change, invalid lines or values would be silently ignored.
Don't check for remote command directory as file storage
This test does not work properly if localstatedir is not “/etc”.
Document master daemon's shutdown behaviour
This wasn't clearly documented until now.
pathutils: Add directory for remote commands
Also add tests to ensure it's never allowed as a file storage path. Aconstant for the lock file is also added.
utils.process.RunResult: Always set "fail_reason" attribute
Add previously missing node daemon GID to getent mock
The UID is there, the GID wasn't.
Add test utility to count calls to function
In some cases it's nice to verify a function has been called exactly Ntimes. This is going to be used in tests for remote commands.
Sort content of generated RPC code
This is similar to commit f5ce761. A stable sort order makes changesbetween versions easier to find.
Add new test for RAPI
Unlike existing tests, this actually tests RAPI at the interface withthe HTTP server. This way authentification can also be tested. A testfor “/2/query/…” is included as it's a bit special.
Add “gnt-job change-priority” sub-command
This can be used to change the priority of a pending or running job (thelatter only if there are unprocessed opcodes).
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Expose changing job priority via LUXI
A new LUXI request is added, in both Python and Haskell.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
jqueue: Allow changing of job priority
This is due to a feature request. Sometimes one wants to change thepriority of a job after it has been submitted, e.g. after submitting animportant job only to later notice many other pending jobs which will beprocessed first. Priority changes only take effect at the next lock...
jqueue: Set task ID for jobs added to workerpool
The job ID is re-used as the task ID, as job IDs are unique.
workerpool: Preserve task number when deferring
When a task is deferred it should receive the same task ID upon beingreturned to the pool.
workerpool: Add method to change task's priority
Using the task ID a pending task's priority can be changed. This will beused to change the priority of jobs in the workerpool.
workerpool: Change data structure for priority change
To prepare for the addition of a new function allowing changing apending task's priority, the internal data structure is slightlychanged. The (optional) task ID is stored as part of the task entry. A...
Automatically enable version-dependent GHC flags
Some GHC flags are very useful, but only appear in more recent GHCversions. To support the use of such flags while still supportingolder compilers, let's add conditional checks and enabling based onthe results....
Added attoparsec unit test for Unicode parsing
Attoparsec is known to have had issues with parsing non-ASCII strings.This test makes sure that parsing of Unicode characters works fine.
Signed-off-by: Michele Tartara <mtartara@google.com>[iustin: small doc string fixes]...
Add "Attoparsec" to the optional haskell packages
This will be needed for the data collectors of the monitoring agent.
Documentation for the NODE_RES level
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Improve make regen-vcs-version
"make regen-vcs-version" is run at every ./devel/upload time, in orderto have reasonably up-to-date information in the uploadedscripts/binaries. However, this means it currently triggersrecompilation of (at least) Ganeti/Version.hs, and sometimes more than...
Fix two hlint warnings
Sorry, I broke lint again :), by introducing two sub-standard changes.
Additionally, this silences an older existing warning that onlytriggers with some versions of hlint (e.g. 1.8.28 which is present inWheezy).
RunCmd: Expose "postfork" callback
The “_postfork_fn” parameter was only used for tests until now. Toimplement a good locking scheme, remote commands must also make use ofthis callback to release a lock when the command was successfullystarted (but did not yet finish)....
Merge branch 'devel-2.6' into master
Conflicts (both trivial): htools/Ganeti/Daemon.hs (_writePidFile rename)...
Merge branch 'stable-2.6' into devel-2.6
Improve error message when migration status fail
Commit 6a1434d7 (“Make migration RPC non-blocking”) changed the APIfor reporting migration status, but has a small cosmetic bug: if themigration status if failure, but the RPC itself to get the statusdidn't fail, it shows the following error message:...
Fix type error in kvm/GetMigrationStatus
Commit 6a1434d7 (“Make migration RPC non-blocking”) changed fromraising HypervisorErrors to returning MigrationStatusobjects. However, these objects don't have an "info" attribute, sothey can't pass a reason back (which is in itself a bug); but the KVM...
Fix PID file writing in Haskell daemons
Currently, the code uses createFile, which has the effect of alwaystruncating the file. This is bad, as the content of the PID file iswiped even when we wouldn't be able to lock it!
We switch to openFd (createFile is just a wrapper over that), and we...
Remove unsafePerformIO usage
We need to change a few things, most importantly CLI options defaults,but otherwise we already used the path to files from functions whichwere already in the I/O monad, so we don't have to change much of thecode flow.
Additionally, Path.hs now has an explicit export list, to stop...
Change keyToFilename/readSSconfFile interaction
Currently, keyToFilename uses itself the default config path. In thelight of making that an function in the IO monad, let's remove the"default" path functionality from this function and make it alwaysrequire the config path; its caller, readSSConfFile, can then do the...
Change type of program options to 'IO [Options]'
Some options have defaults that depend on the environment, and wecould handle these in two ways:
- use a place-holder value (e.g. data X a = Default | Custom a) that is later read from the environment- move the options list to IO monad, where it can read the...
Two small logging improvements
The "starting" message is changed to match the Python one, and in casethe preparation fails, we also log the error (beside printing it onstderr or writing it to the error reporting pipe), as at this timelogging is usually set up....
Annotate confd startup/luxi binding error
The luxi binding is now annotated for better readability.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Switch writing of PID file to new reporting style
Currently, the writing of the PID file uses a "standard" errorreporting: catch exception, transform it into a 'Result' type, leavehandling of that to the caller. However, for daemon startup, weactually want exceptions to be propagated up until the handler which...
Move and rename BasicTypes.annotateIOError
It turns out that annotateIOError already exists in the standardlibrary, with a different purpose (this made me waste 10 minutestrying to understand why the types were wrong…). Let's move this outof BasicTypes, since it's used only in Daemon.hs, and also rename it....
Switch daemon startup to pipe-based error reporting
This changes the daemon reporting error to the same mechanism as inPython: use a pipe which is written to from the forked children.
Move preparation steps of confd in prepMain
This does the address parsing earlier, before we fork, forbetter/faster error reporting.
Split queryd run into prepare and exec
This will help with the general daemon split of prepare/run, and flagerrors earlier in the startup.
Convert tag objects to a safer type
Currently, we keep information about the "target" of a tag operationin a data type similar to (TagKind, Maybe String). This is unsafe, asnothing (at the type level) prevents us from accidentally having(TagCluster, Just "instance1.example.com"), or (TagInstance, Nothing)....
Switch Luxi TH code from simple to custom fields
This is needed so that we have more flexibility in generating Luxiserialisation code (deserialisation is still custom). Also, onlyexceptions are now using the 'simple' field types, so we might be ablelater to convert and remove that TH code as well....