Fix the broken unittests
Fix the unittest that have been broken when we lifted the Disk objectsfrom the Instances.
Signed-off-by: Ilias Tsitsimpis <iliastsi@grnet.gr>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Implement getDisks in Confd
Add 'ReqInstanceDisks' request type and allow Confd to query for thedisks of an instance. The helper function 'getInstanceDisks' returns thelist of instances on the given node along with their disks and is usedby the function 'addInstNameToLv'....
Add timestamp/serial_no slot to disk objects
Now that disks are top level citizens in config,they need a timestamp and a serial_no slot.
Add disks entry to config.data
Add disks entry to config.data.
Add basic test for CPU speed
In this example, the cluster has two nodes and four instances, two withprimary on each of the nodes. The most scarce resource on this clusterare (virtual) CPUs and the second node has 3 times the CPU speed of thefirst one. So distributing the instances 1 and 3 gives a more balanced...
Add VTypeFloat
...in order not to have to declare floating pointvalues as VTypeInt and rely on the sloppiness ofthe JSON specification to not distinguish betweenintegers and floating point numbers.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Add optional fields for job livelocks and process IDs
This will allow to check if a particular job is alive, and send signalsto it when it's running.
The fields aren't serialized, if missing, for backwards compatibility.
Signed-off-by: Petr Pudlak <pudlak@google.com>...
Clean up from LockAllocation what is no longer used
With the change from LockAllocations to LockWaitings, severalmanipulation operations had to be implemented for LockWaitingsand became unused in LockAllocation. Remove these functions thatare no longer used....
Verify the answer of opportunistic lock union
Verify that the result set of the opportunistic lockunion is correct.- If a lock is mentioned in the result set, the request must have been granted.- If it is not mentioned, the owner state must be unchanged....
Verify that opportunistic lock union is monotone
Verify that a request to opportunistically allocate locks neverreduces the amount of locks held.
Use intersection and union for arbitrary waitings
Allow intersection and opportunistic lock union in the constructionof the arbitrary element of a lock waiting. While definable fromupdateLocks, they are typical interface functions and can helpget a more natural distribution in arbitrary....
Verify: liftM extRepr . readJSON . showJSON = Ok . extRepr
In other words, verify that for LockWaiting, the JSON encodingcan be decoded correctly up to observable indistinguishability.
Verify readJSON . showJSON = Ok for LockRequest
Verify that the JSON encoding of a LockRequest can correctlybe decoded.
Verify extRepr-equal states are equal on updateLocksWaiting
Add a test that verifies that extRepr-equal states cannot bedistinguished by updateLocksWaiting-transitions. To obtainextRepr-equal states, we compare each state to the one computedfrom its extensional representation....
Verify that extRepr-equal states behave equal on updateLocks
Add a test that verifies that extRepr-equal states cannot bedistinguished by updateLocks-transitions. To obtain extRepr-equalstates, we compare each state to the one computed from itsextensional representation....
Verify decoding from extensional representation is sound
Verify that, on the image of extRepr, it holds extRepr . fromExtrRepr = id.In other words, verify that when obtain a lock waiting from an extensionalrepresentation, it has the same extensional representation....
Verify that all pending requests are justified
If a request is pending in a lock waiting structure, it has to fulfilltwo properties:- It must be a valid request, not violating any lock order requirements, and- it must not be possible to fulfill it immediately....
Verify soundness of notifications
Verify that, if a lock owner gets notified, he- had a pending request, and- the request is fulfilled now.
Strengthen progress property in lock waiting
Not only verify that after the blockers for one pending requesthave gone, some request was honored, but also verify that it wasnot a request by one of the blockers.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Add process id to lock-owner description
...so that we can notify owners when their pending request got granted.
Verify that some progress is made after lock releases
Add a basic sanity check for progress on pending requests. If arequest is pending and all owners it is blocked on release theirlocks, there is at least one pending request that can be granted(the said one). So verify that at least one owner gets notified...
Verify that an owner gets notified
Add a basic sanity check on notification: if a request is blocked,and all old pending owners and blockers give up their resources,the owner gets notified of his lock being granted.
Verify that pending requests get fulfilled eventually
Add a basic sanity check for the lock waiting mechanism. If a requestget added to pending, and all blockers and pending owners give up theirresources it is automatically handled.
Export requestSucceeded
...so that this predicate can be used for testing the LockWaitingstructure as well.
Verify that unfulfilled requests lead to pending requests
Verify that an owner has a pending request after a waiting requestnot fullfilled immediately.
Verify locks can't be updated while having a pending request
Add a verify first sanity check for lock waiting: a user that has a pendingrequest cannot modify his locks. This patch also brings in the necessaryinfra structure for having arbitrary waiting structures....
Export the test locks for testing
As lock waiting is also based on locking, export thetest lock/owner structure, so that the tests for lockwaiting can use them.
Fix typos in comments
Add zeroing timeout params to opcode
This patch adds two parameters controlling the zeroing timeout - onethat is fixed and another that depends on the amount of data (size ofdisks) to zero.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Add the zeroing-image option
This patch adds the zeroing-image option to gnt-cluster and theOpBackupExport params. The many changes are all minor, yet necessary.
Add the zero-free-space option
Although unused for now, add the zero-free-space option to the backupexport opcode, and add a rudimentary check for whether it should beused.
Verify readJSON . showJSON = Ok for ClientType
Verify that the JSON encoding of the ClientType can correctlybe decoded.
Verify soundness of listAllLocksOwners
Verify that every owner mentioned in the output of listAllLocksOwnersactually owns the locks claimed.
Verify that all locks are mentioned in listAllLocksOwners
Add a test verifying that whenever an owner owns a lock, itis also mentioned in the output of listAllLocksOwners.
Verify consistency between listAllLocks{,Owners}
As listAllLocks is supposed to list all locks currently in use, this shouldbe a superset listAllLocksOwners, i.e., of all locks actively owned by someuser.
Verify that the list of all locks is complete
Verify that all locks owned by someone are included in thelist of all (active) locks.
Fix lint errors after moving Luxi code to UDSServer
Remove unused imports and update the 'buildCall' call in tests.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Identify lock owners by either string or job id
Most requests asking for resources are jobs. However, in exceptionalcases, other requests (like currently requests to masterd requiring theconfiguration) need to ask for resources. They identify themselves by...
Add tests for (de)serialization of ClientId
.. to test if `readJSON . showJSON = Ok`.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Verify serialisation of LockLevel
For the new JSON instance Ganeti.Locking.Locks.LockLevel, verify thatreadJSON . showJSON = Ok.
Add network locks to Haskell
Ganeti also has a lock level for networks, however this level was forgottenwhen the Ganeti lock hierarchy was implemented in Haskell.
Add a test for (de)serialization of Ssconf
.. to verify that readJSON . showJSON = Ok.
Create a data type for representing RPC compressed data
It is serialized as a JSON pair. The first element is a numberdescribing the compression type, the second element the actual data(in this case Base64+Zlib encoded).
On the Python side it is decompressed by backend._Decompress....
Refactor warnings issued by the latest 'hlint'
.. as the newest version of hlint adds additional checks
Verify that all lock owners indeed own a lock
Add a test verifying that the list of lock owners is nottoo big in the sense that every subject contained in thatlist owns at least one lock.
Verify the list of lock owners is complete
In other words, verify that if all the lock owners release theirlocks, the empty lock allocation is reached.
Add the interruptible option to the delay opcode
To prepare for the use of sockets to interrupt a running delay opcode,this patch adds an option to the opcode for doing so.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Test the instance 'ArrayObject Disk'
.. which is one of the few ones that use extra fields.
Add a helper function for testing 'ArrayObject' instances
It checks if 'fromJSArray . toJSArray' is equivalent to 'Ok'.
Verify the intervall property of GanetiLocks
According to the specification in our design, if A is a grouplock and B belogs to this group, then all locks between A and Bin the lock order also must belong to A. Verify this.
Verify implication restriction on locks
As specified in our design, if A is a group lock and Bbelongs to A, then A must be earlier in the lock orderthan B. Verify this property.
Update Arbitrary GanetiLocks
With the extension of Ganeti.Locking.Locks.GanetiLocks to thefull lock hierarchy, also extend the instantiation of Arbitrary.
Allow cluster mac prefix modification
Extend LUClusterSetParams to allow the modification of the clustermac-prefix setting in 'gnt-cluster modify' command.
This fixes part of issue 239.
Signed-off-by: Dimitris Bliablias <bl.dimitris@gmail.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Verify serialization/deserialization consistency for Locks
As we made Ganeti.Locking.Locks an instance of JSON, we better verifythat we can readJSON what we showJSON.
Add test for serialization of LockAllocation
Verify readJSON . showJSON = Ok for LockAllocations.
Verify the result of opportunistic union
Verify that the result returned by the opportunistic union correctlyreflects the state change: locks not in the result are not changed,locks in the result are as requested.
Verify that opportunistic union only adds to the locks held
Add a test verifying that by opportunistic union the set of locksheld, and the level at which the locks are held, only increases.
Use opportunisticUnion in Arbitrary LockAllocation
Also allow the newly exported function opportunisticLockUnion in theconstruction of arbitrary lock allocations.
Generate arbitrary :: LockAllocation according to interface
While we know that freeLocks and intersectLocks are defined in termsof updateLocks, it is still cleaner to test against the interface: themodule Ganeti.Locking.Allocation exports several functions, and all...
Add a test verifying that lock-implication is honored
Locks can be included in one another. This inclusion can be violatedin two ways, viz.,- A holds a lock L and B holds an exclusive lock L belongs to, and- A holds an exclusive lock and B holds a lock L belongs to....
Verify that every owner mentioned as blocking actually is
If a request is blocked by multiple lock owners, verify that eachsingle one of them actually blocks the request. In other words,verify that, whenever all but one release their lock, the requeststill does not succeed....
Verify that the set of blocking owners is big enough
When a request is blocked, the list of blocking ownersshould exhaust all blocking reasons, i.e., if thoseowners release all their locks, the update must succeed.
Make TestLock an instance of Lock
Equip the type of test locks with a simple group structureconsisting of two lock groups and a big lock.
Verify that releasing a lock always succeeds
There are no restrictions on releasing a lock; so this operationhas to succeed unconditionally. Verify this.
Add '-c | --instance-communication' flag to instance modify
Make the lock allocation part of WConfD's state
As WConfD is supposed to be the authoritative source forconfiguration and locks, it needs to have access to the lockstate as well.
Verify atomicity of lock updates
Add a test verifying the atomicity of an update, i.e., if an updatesucceeds, then all requests must be satisfied in the final state, andif it fails, the state must be unchanged.
Verify lock stability over updates
Add a test that verifies that updates by one user do notaffect the locks held by a different user.
Verify Basic property for lock allocation
Verify the minimal consistency property for any formof lock handling: if a user holds an exclusive lock,then no other user can hold the same lock (neitherexclusively, nor shared).
Add mtime/ctime to ConfigData
It's used on the Python side, it's been missing on the Haskell side.
Add the WConfD daemon to build configuration files
Also list it in the Haskell datatype, constants, Python constants andtest configuration.
Add tests for Ganeti.THH.Types
In particular, test (de)serialization of "OneTuple".
Add a helper test method for checking deserialization
It tests if a deserialization method fails properly when an invalidinput is given.
Add 'instance_communication_network' to 'OpClusterSetParams'
... and update the unit tests.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
OpCodes: modify InstanceReinstall for private, secret params
Modify InstanceReinstall to accept and process private and secretparameters.
Signed-off-by: Santi Raffa <rsanti@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
OpCodes: modify InstanceCreate for private, secret params
Modify InstanceCreate to accept process private and secret parameters.
OpCodes: modify ClusterSetParams for private parameters
Modify ClusterSetParams to accept and process private parameters.
OpCodes: modify InstanceSetParams for private parameters
Modify InstanceSetParams to accept and process private parameters.
Add private OS parameters to cluster and instance conf
This updates objects, constructors and mocks for Instance and Clusterobjects in Python and Haskell.
Add Private types to Python, Haskell
This commit adds the private containers to Python and Haskell.
serializer: emit and encode Private values
For inbound data the simplest, safest thing to do is to traverse allJSON right after encoding and search for private parameters by key.
This ensures that all consumers of this data get Private valuestransparently and consistently; the serializing methods don't have to...
OpCodes test: fix argument order (expected/but got)
Fix expected/but got value in some unit tests.
Add CLI and OpCode instance communication option
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Add metadata daemon
... including, user, group, daemonizing code with command lineoptions, integration with the Snap HTTP server, and logic.
Merge branch 'stable-2.11' into master
Add andRestArguments to IDiskParams
In this way, we cann pass through the opaque parametersrequired for disk creation and modification in the case ofexternal storage.
Add genAndRestArguments :: Gen (Map String JSValue)
So that objects using AndRestArguments are available for testing.As the AndRestArguments are intended for passing through additionalparameters passed on the command line, we restrict them to thevalues that are allowed at this position: Strings of ASCII characters...
Update getDirJobIDs to use ResultT
Also simplify code and remove unused functions.
Generalize type signatures of functions that produce Result
This allows them to be used in any 'MonadError', in particular in monad stackscomposed using ResultT, without explicit lifting.
Add 'provider' to IDiskParams
IDISK_PROVIDER was included in python's IDISK_PARAMS, so itshould also be included in the Haskell code.
Now that luxid creates and enqueues jobs, without this patch theExtStorage interface is broken as the user can not pass the disk...
Add opcode parameter for the maximal number of running jobs
This parameter of OpClusterSetParams will allow to set themaximal number of jobs to be run simultaneously.
Use ClockTime instead of Double in fields in Objects.hs
This affects "mtime" and "ctime" fields in all data types.
This also forces explicit declaration of how the fields are serializedin Query.
Also fix MaybeForJSON parsing and add unit tests
Its readJSON was mistakenly implemented as an infinite loop.
Add (de)serialization unit tests for TimeAsDoubleJSON
This includes adding an Arbitrary instance for TimeAsDouble, and also anorphan instance for Arbitrary ClockTime so that the current tests workunmodified.
Unit tests for KVM daemon
Add unit tests for KVM daemon.
KVM daemon datatype, user and group
Verify incoming RPCs against candidate map
From this patch on, incoming RPC calls are checked againstthe map of valid master candidate certificates. If no mapis present, the cluster is assumed to be inbootstrap/upgrade mode and compares the incoming call...
Merge branch 'stable-2.10' into master