Helga Velroyen [Tue, 20 Nov 2012 17:16:29 +0000 (18:16 +0100)]
Check ispecs against ipolicy on instance modify
When modifying an instance, so far the specs were not checked against
the ipolicy. This patch fixes this issue.
Note that for backend parameters which have a minimum and a maximum
value (currently only memory), it checks both limits against the
ipolicy. Because locking of the instance's node group was necessary, a
TODO of commit
b8925b86 was fixed as well.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Mon, 19 Nov 2012 21:34:12 +0000 (22:34 +0100)]
locking.LockSet: Replace boolean parameter with constants
Upcoming changes will add opportunistic locking to “locking.LockSet”.
Doing so will require additional code in “LockSet.__acquire_inner”, at
which point the existing “want_all” parameter does not always apply.
This patch replaces it with a “mode” parameter. For opportunistic
locking a new mode will be added.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 21 Nov 2012 05:14:32 +0000 (06:14 +0100)]
RAPI: Clarify non-exhaustive nature of examples
The examples given in the RAPI documentation are not exhaustive. This
patch tries to clarify this by adding ellipses where appropriate. One
example's indentation is also fixed.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 21 Nov 2012 05:07:01 +0000 (06:07 +0100)]
Add tests for __repr__ in locking classes
“locking.PipeCondition” and “locking.SharedLock” define “__repr__”,
which until now was not tested at all.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 21 Nov 2012 04:57:22 +0000 (05:57 +0100)]
test/*.py: s/'/"/
Now that 2.6 is essentially finished and 2.7 going to be branched
soon-ish, I thought it would be a good moment to replace some single
quotes in test/*.py. Merge pains should be limited.
In one place in test/ganeti.locking_unittest.py, spaces are added for
assignments.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 21 Nov 2012 01:42:17 +0000 (02:42 +0100)]
network: Style updates
- Quote characters
- Docstrings
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 21 Nov 2012 01:42:00 +0000 (02:42 +0100)]
Document requirement for ipaddr and bitarray modules
Also update configure.ac.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 21 Nov 2012 04:08:09 +0000 (05:08 +0100)]
ht: Complain if TNone isn't listed first for TOr/TAnd
Some type descriptions are rather long. If "None" is listed at the end
or somewhere in between it is easily missed. Therefore it should be at
the beginning, e.g. "None or (long description)".
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 21 Nov 2012 03:52:06 +0000 (04:52 +0100)]
Add unit test for default parameter default values
Fails if the default value of an opcode parameter doesn't verify.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 21 Nov 2012 04:06:08 +0000 (05:06 +0100)]
opcodes: Network parameter improvements and fixes
- Use variables for checks for IP addresses (makes them more consistent,
too)
- Fix checks to accept default values
- Small description improvements (e.g. s/IP/IP address/)
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 21 Nov 2012 05:51:40 +0000 (06:51 +0100)]
Merge branch 'devel-2.6'
* devel-2.6:
Fix opcode validation for OpOobCommand.command
Fix typo in manpage for gnt-os
gnt-instance.8: Add index to disk and nic removal
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 21 Nov 2012 03:52:29 +0000 (04:52 +0100)]
Merge branch 'stable-2.6' into devel-2.6
* stable-2.6:
Fix opcode validation for OpOobCommand.command
Fix typo in manpage for gnt-os
gnt-instance.8: Add index to disk and nic removal
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Mon, 19 Nov 2012 21:09:47 +0000 (22:09 +0100)]
locking: Use frozenset instead of utils.UniqueSequence
In this case “frozenset” is good enough as the result's order doesn't
matter--it is the input to “sorted” (“utils.UniqueSequence” preserves
the order). “frozenset” is ca. 25% faster for this use-case.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Tue, 20 Nov 2012 03:19:12 +0000 (04:19 +0100)]
locking: Add test for downgrade without names
Until now there was no test for calling “LockSet.downgrade” without
specifying any names.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Iustin Pop [Tue, 20 Nov 2012 18:00:13 +0000 (19:00 +0100)]
Fix some more hlint warnings
Sorry…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Iustin Pop [Tue, 20 Nov 2012 16:44:26 +0000 (17:44 +0100)]
Fix tools/move-instance after networks addition
Commits
d4117a72 (“Modify LUInstanceCreate to support networks”) and
09a3d21f (“Pass detailed network info in hooks“) expanded the nic
information as computed by _NICToTuple, which means that the format of
the result value from LUInstanceQueryData has changed, and
move-instance has not been changed to deal with this.
A proper fix needs to support correctly importing from old clusters,
right now this is a temporary fix to ensure QA still passes.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Dimitris Aragiorgis [Tue, 30 Oct 2012 15:51:42 +0000 (17:51 +0200)]
Move _BuildNetworkEnv() at top-level
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Tue, 30 Oct 2012 11:00:57 +0000 (13:00 +0200)]
Use bitarray 0.8 interface
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Tue, 30 Oct 2012 01:31:02 +0000 (03:31 +0200)]
Minor fix in query.py
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Tue, 30 Oct 2012 01:10:31 +0000 (03:10 +0200)]
Minor fix in ovf
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Mon, 29 Oct 2012 19:11:59 +0000 (21:11 +0200)]
Fix a few docstrings
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Mon, 29 Oct 2012 19:00:06 +0000 (21:00 +0200)]
Simplify GenerateFree in network module
GenerateFree now returns the first available IP in the network
or raises AddressPoolError if it is full.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Mon, 29 Oct 2012 18:39:58 +0000 (20:39 +0200)]
Replace string values with proper constants
_UnlockedCommitIp is used either for releasing or reserving an
IP inside a network. New constants RELEASE_ACTION/RESERVE_ACTION
are used to decide which is the case.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Mon, 29 Oct 2012 18:31:54 +0000 (20:31 +0200)]
Add docstring to the GetECReserved config method
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Mon, 29 Oct 2012 18:10:04 +0000 (20:10 +0200)]
Simplify mac generation funtions
Introduce new config method _UnlockedGetNetworkMACPrefix and refactor
original _GenerateOneMAC so that it takes prefix as an optional
argument and returns a function that actually generates a MAC. If
prefix is not given or None it uses the cluster level default.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Tue, 16 Oct 2012 05:46:51 +0000 (08:46 +0300)]
Fix _ComputeNics to include network slot
This bug was introduced after rebasing master branch.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Fri, 12 Oct 2012 18:37:41 +0000 (21:37 +0300)]
Fixes to pass py-apidoc (make commit-check)
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Fri, 12 Oct 2012 14:49:06 +0000 (17:49 +0300)]
Fixes to pass pep8 (make lint)
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Fri, 12 Oct 2012 11:18:28 +0000 (14:18 +0300)]
Fixes to pass pylint (make lint)
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Thu, 11 Oct 2012 13:30:04 +0000 (15:30 +0200)]
Fixes to pass unittests (make check)
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Wed, 3 Oct 2012 15:35:42 +0000 (18:35 +0300)]
Check for existing networks/groups in Network LUs
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Fri, 27 Jul 2012 07:50:04 +0000 (10:50 +0300)]
Add tags in network objects
Support:
- gnt-network add --tags.
- gnt-network list-tags/add-tags/remove-tags/.
- gnt-network list -o +tags
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Mon, 18 Jun 2012 16:05:28 +0000 (19:05 +0300)]
Change default fields for network listing
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Thu, 7 Jun 2012 15:51:34 +0000 (18:51 +0300)]
Pass detailed network info in hooks
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Wed, 6 Jun 2012 12:57:37 +0000 (15:57 +0300)]
Pass netinfo in rpcs
If a nic has a network field then encapsulate a network object in
netinfo slot for every rpc. This is needed to pass network info to
scripts managing nics (kvm-vif-bridge).
Introduce _BuildNetworkEnv().
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Tue, 5 Jun 2012 23:08:11 +0000 (02:08 +0300)]
cfgupgrade for networks
Add networks slot in config data.
Add networks slot in nodegroups.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Tue, 5 Jun 2012 23:03:08 +0000 (02:03 +0300)]
Use network's mac prefix
Modify existing GenerateMAC so that it takes network as an argument.
Substitute _GenerateOneMAC with a decorator _GenerateMACPrefix that
chooses the prefix (network's or cluster's) and wraps
_GenerateMACSuffix that generates the three remaining bytes.
Check mac prefix validity when adding/modifing a network.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Apollon Oikonomopoulos [Mon, 4 Jun 2012 20:52:31 +0000 (23:52 +0300)]
Include networks in Makefile and bash completion
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Apollon Oikonomopoulos [Mon, 4 Jun 2012 20:51:44 +0000 (23:51 +0300)]
Add gnt-network man page
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Mon, 4 Jun 2012 20:20:09 +0000 (23:20 +0300)]
Rapi support for networks
Support:
- GetNetwork(s)
- CreateNetwork
- ConnectNetwork
- DisconnectNetwork
- RemoveNetwork
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Apollon Oikonomopoulos [Mon, 4 Jun 2012 20:15:42 +0000 (23:15 +0300)]
ss_conf support for networks
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Mon, 4 Jun 2012 20:09:27 +0000 (23:09 +0300)]
Introduce client support for networks
gnt-network is used to manipulate and handle networks that currently
provides the following operations:
* Add a new network:
gnt-network add --network=1.2.3.0/28 --gateway=1.2.3.1
--add-reserved-ips=1.2.3.4,1.2.3.5 testnet
* Remove an existing network:
gnt-network remove testnet
* Modify an existing network:
gnt-network modify --gateway=1.2.3.6
--network-type=private
--network6=2001:648::/64
--gateway6=none testnet
--add-reserved-ips=1.2.3.10,1.2.3.10,
--remove-reserved-ips=1.2.3.20
testnet
* Connect an existing network to a nodegroup:
gnt-network connect testnet default bridged br100
gnt-network connect testnet <nodegroup> <mode> <link>
(pass all for <nodegroup> to connect to all nodegroups)
* Disconnect an existing network from a nodegroup:
gnt-network disconnect testnet <nodegroup>
(pass all for <nodegroup> to disconnect from all nodegroups)
* List available networks:
gnt-network list
* Show network info:
gnt-network info [testnet]
Introduce new option NOCONFLICTSCHECK_OPT for not checking for
conflicting IPs. Using this might cause data inconsistency.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Apollon Oikonomopoulos [Mon, 4 Jun 2012 20:03:44 +0000 (23:03 +0300)]
Modify instance client to support networks
Add --no-conflicts-check option.
Modify instance query methods to return nic's network info.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Apollon Oikonomopoulos [Mon, 4 Jun 2012 19:58:12 +0000 (22:58 +0300)]
Peripheral changes for networks
Specifically:
* export a new environment variable for the gnt-os-interface
* export a new environment variable for KVM specific scripts
* ovf support for networks
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Mon, 4 Jun 2012 19:39:53 +0000 (22:39 +0300)]
Modify LUInstanceSetParams() to support networks
Keep backwards compatibility just like LUInstanceCreate() and support
network parameter for NIC object.
Add ec_id argument in Update() to be able to commit temporary IPs that
were previously reserved.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Mon, 4 Jun 2012 19:37:12 +0000 (22:37 +0300)]
Modify RemoveInstance() to support networks
Release any IPs held the instance before actually removing.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Mon, 4 Jun 2012 19:31:30 +0000 (22:31 +0300)]
Modify LUInstanceCreate to support networks
Implement backend support, to export the IP pool management
functionality to the clients. When the new NIC parameter 'network' is
given, the ippool management system is triggered. If a NIC belongs to
a network, it inherits the netparams (mode, link) as its nicparams. If
a network is requested, then no 'mode' or 'link' should be
specified. Backwards compatibility is preserved, if 'network'
parameter is omitted.
Examples for IP pool management system:
--net 0:network=net1
for an IPless NIC in a network
--net 0:ip=pool,network=net1
for automatically generated IP inside the network
--net 0:ip=1.2.3.4,network=net1
for specific IP inside the network
--net 0:network=none
for IPless NIC with default nicparams
Examples for traditional assignment:
--net 0:ip=1.2.3.4,link=br100,mode=bridged
--net 0:ip=none,link=rt2,mode=routed
--net 0
Check for conflicting IPs to ensure IP uniqueness inside nodegroups.
IAllocator should be extended to become network aware. If an instance
has a NIC inside a network, then IAllocator must pick a node that its
nodegroup is already connected with that network.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Mon, 4 Jun 2012 19:24:47 +0000 (22:24 +0300)]
Config methods for reserving/releasing IPs
Use TemporaryReservationManager for IP handling to ensure config data
consistency.
Implement:
- methods for reserving/releasing IPs
- _UnlockedCommitTemporaryIps()
- CheckIPInNodeGroup() used for conflicting IPs
- GetGroupNetParams() used for inheriting netparams.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Mon, 4 Jun 2012 18:08:36 +0000 (21:08 +0300)]
Implement network/nodegroup mappings
LUNetworkConnect() connects a network to a nodegroup and defines
its netparams (mode and link). Specifically:
* Check if network already exists
* Check if netparams are valid
* Check if already connected to the same nodegroup
* Produce errors if any conflicting IPs found
* Update the 'networks' slot in NodeGroup config object
with the network UUID and netparams requested. E.g.
nodegroup_uuid {
networks {
network_uuid: {mode: 'bridged', link: 'br100'}
}
}
LUNetworkDisconnect() unmaps a network from a nodegroup:
* Produce errors if an instance has a NIC with IP inside the network
* Remove corresponding entry in 'networks' slot of NodeGroup object
Check for conflicting ips can be avoided with corresponding option.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Mon, 4 Jun 2012 18:05:03 +0000 (21:05 +0300)]
Implement LUNetworkSetParams
Support modifying all network parameters except for network (ip
range). Cannot modify gateway and reserved ips at the same time.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Apollon Oikonomopoulos [Mon, 4 Jun 2012 18:02:55 +0000 (21:02 +0300)]
Implement LUNetworkQuery
Summarily list all existing networks
Supply detailed info for every existing network
- List used/free IPs
- List instances with NICs assigned to the corresponding network
- List NIC index and IP for the above instances
Implement complementary config methods for retrieving networks.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Mon, 4 Jun 2012 15:54:02 +0000 (18:54 +0300)]
Basic IP pool management logic
Implement LUs for corresponding opcodes:
* LUNetworkAdd:
- Check for IP validity
- Reserves all necessary IPs
- Create new Network config object
* LUNetworkRemove:
- Checks if connected to any nodegroup
- Remove a Network config object
Implement basic config methods:
* LookupNetwork()
- Given the network name return the network UUID
* AddNetwork()
- Add a new network to the config
* RemoveNetwork()
- Remove a network from the config
Add new locking level: LEVEL_NETWORK
Add various useful config methods for retrieving network info.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Apollon Oikonomopoulos [Mon, 4 Jun 2012 14:33:22 +0000 (17:33 +0300)]
Introduce new module for IP pool management
Add new library module lib/network.py.
Introduce new class: AddressPool.
AddressPool implements all operations needed for managing IPs inside
the IP pool.
Given a Network config object (nobj), the class:
* initializes the corresponding IP pool object via
network.AddressPool.InitializeNetwork(nobj)
* obtains the corresponding IP pool object via
network.AddressPool(nobj)
* manipulates IPs inside the pool
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Dimitris Aragiorgis [Mon, 4 Jun 2012 13:50:42 +0000 (16:50 +0300)]
IP pool related objects, opcodes and constants
Config objects:
* Introduce L{Network} with
- IPv4 network field (mandatory)
- IPv4 gateway, IPv6 (network/gateway), mac prefix, type (optional)
* Modify existing config objects to support networks:
- Add new slot 'network' to L{NIC} config object
- Add new slot 'networks' to L{NodeGroup} config object
Opcodes:
* Introduce new opcodes for networks
- add/remove/modify/query/connect/disconnect.
* In InstanceCreate/InstanceSetParams add conflicts_check option
Constants:
* INIC_PARAM 'INIC_NETWORK'
* NIC_IP_POOL for automaticaly obtain an IP from a pool
* NETWORK_TYPE_PUBLIC/PRIVATE for network types
Checking of network_type handled by the opcode parameter validation.
Introduce _CheckCIDR*Notation() functions for network parameters
validation.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Thu, 15 Nov 2012 12:24:14 +0000 (13:24 +0100)]
Add more basic validation types
This mirrors the ht.py types PositiveInt, NonNegative, etc., except
that they work at a more generic level (any numeric type, respectively
any non-empty list).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>
Iustin Pop [Tue, 20 Nov 2012 09:59:45 +0000 (10:59 +0100)]
Fix a hlint warning
This is yet-another version-specific warning (fixed in later hlint
versions), but it's better to keep it clean.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michele Tartara [Mon, 19 Nov 2012 14:20:36 +0000 (15:20 +0100)]
Add DRBD parser unit tests
This adds tests that existing test files can be parsed by the Haskell
parser as well, plus one new test file.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michele Tartara [Mon, 19 Nov 2012 14:19:51 +0000 (15:19 +0100)]
Add parser for DRBD /proc file
A new directory for haskell modules about block devices has been created
The parser is divided in two modules:
* one exports the data types describing the DRBD status
* one exports the parser itself
Signed-off-by: Michele Tartara <mtartara@google.com>
[iustin@google.com: indentation/alignment fixes]
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Tue, 6 Nov 2012 14:23:34 +0000 (15:23 +0100)]
Properly log errors when setting up daemon FDs
While writing the pipe-based reporting and trying various ways to
break the startup, I fought for a while trying to understand why error
reporting was _different_ when running the daemon as a user (with no
rights). It turns out that setupDaemonFDs wants to open the log file
in append mode way before, so we are not protected by the 'prepare'
phase.
This patch explicitly runs the 'setupDaemonFDs' function under the
same handler as the prepare phase, with the only change that here we
instruct handlePrepErr to not log the message via log*, since logging
is not yet set up.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 19 Nov 2012 09:21:28 +0000 (10:21 +0100)]
Switch opcode data type from normal to record constructors
Currently, the OpCode definitions are using normal constructors:
data OpCode = OpTestDelay Double Bool [String]
| OpInstanceFailover String Bool (Maybe String)
…
While this works for a few opcodes, it becomes unwieldy when dealing
with a bigger number of opcode definitions and/or with opcodes having
many fields.
This patch changes the opcodes to record-based constructors, so that
we get for free accessor functions:
data OpCode
= OpTestDelay {
opDuration :: Double,
opOnMaster :: Bool,
opOnNodes :: [String]
}
| OpInstanceFailover {
opInstanceName :: String,
opIgnoreConsistency :: Bool,
opTargetNode :: Maybe String
}
…
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>
Iustin Pop [Mon, 19 Nov 2012 16:07:01 +0000 (17:07 +0100)]
Fix docstrings for the Filter type
While looking at the opcode docs and clicking accidentally on the
filter type, I saw that the haddock formatting is broken due to
non-escaped use of special chars. Let's convert the ascii-like
formatting to haddock, and have nicer apidoc.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 19 Nov 2012 15:57:24 +0000 (16:57 +0100)]
Simplify a few test helpers
Use more monadic combinators instead of explicit code.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>
Iustin Pop [Fri, 16 Nov 2012 13:24:20 +0000 (14:24 +0100)]
Add custom code for CV_E* constants to convert-constants
Currently, the cluster verify errors are defined as follows:
CV_ECLUSTER_FOO = (TCLUSTER, "ECLUSTER_FOO", "description")
This means there's no standalone name for the string "ECLUSTER_FOO",
which makes it hard to derive automatically a type for this union in
Haskell.
There are three possible fixes:
- manually separate the CV_ECLUSTER_FOO_STR = "ECLUSTER_FOO" in
constants.py
- manually extract the strings in Haskell code
- change convert-constants to automatically export virtual constants
for those
After discussion on IRC, I've taken the latter approach; even though a
bit hack-ish, it avoids manual work and potential errors.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Mon, 19 Nov 2012 09:10:09 +0000 (10:10 +0100)]
Improve error message for wrong opcode parameter values
When given an empty string as value to a parameter that doesn't accept
empty strings, the error message was confusing, since it was showing
just as a double space.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>
Iustin Pop [Thu, 15 Nov 2012 09:28:17 +0000 (10:28 +0100)]
Cleanup ht's use of positive/strictpositive
Currently, ht.py uses a bad terminology for positive/non-negative
numbers. Per http://en.wikipedia.org/wiki/Positive_number, this is the
correct terminology:
- A number is positive if it is greater than zero.
- A number is negative if it is less than zero.
- A number is non-negative if it is greater than or equal to zero.
- A number is non-positive if it is less than or equal to zero.
So this patch renames things as follows:
- TPositiveInt ⇒ TNonNegativeInt
- TStrictPositiveInt ⇒ TPositiveInt
- TMaybePositiveInt ⇒ dropped, not used anywhere
- TMaybeStrictPositiveInt ⇒ TMaybePositiveInt
- TPositiveFloat ⇒ TNonNegativeFloat
- TStrictNegativeInt ⇒ TNegativeInt
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 13 Nov 2012 15:37:31 +0000 (16:37 +0100)]
Split OpCode.hs and add module for opcode parameters
Due to TemplateHaskell stage restrictions, we can't define parameters
in the same module as we're using them for TH, so we have to define
all module parameters in a separate module.
This patch therefore splits OpCodes.hs in two, adding that module and
moves most code there (types, parameters, etc.). The remaining parts
in OpCodes.hs, the actual opcode definitions, now use more parameters
instead of direct field definitions (more will come later)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>
Iustin Pop [Thu, 15 Nov 2012 09:05:45 +0000 (10:05 +0100)]
Create a new Ganeti/Types.hs module
There are already three cases where we copied type definitions between
the htools-specific types into the main ganeti code. Let's stop doing
this ☺ and create a common types module that holds these.
Note that there already exists BasicTypes.hs, but that refers to very
low-level types, and can't use TH derivation itself.
A side effect of this unification is that there is a small conflict
between AdminStatus/AdminOffline and InstanceStatus/AdminOffline. As
such, I renamed AdminOffline and AdminDown to StatusOffline/StatusDown
in the InstanceStatus type.
The patch also moves the tests related to these types to a new test
module.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>
Michael Hanselmann [Mon, 19 Nov 2012 16:14:44 +0000 (17:14 +0100)]
locking: Simplify condition
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 14 Nov 2012 10:59:04 +0000 (11:59 +0100)]
configure: Shell cleanup
- Remote unnecessary quote characters
- Change some uses of “test "$foo"” to “test -n "$foo"”
- Merge “if/else/if” into “if/elif”
- Wrap lines longer than 80 characters and add a test to “make check”
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Iustin Pop [Mon, 19 Nov 2012 12:34:42 +0000 (13:34 +0100)]
Fix opcode validation for OpOobCommand.command
The 'command' attribute of the OpOobCommand command is defined with a
default value of None, but its validation requires a member of
constants.OOB_COMMANDS, which doesn't accept None. This result in the
following error when submitting an opcode without the command:
error type: wrong_input, error details:
Parameter 'OP_OOB_COMMAND.command' fails validation
I suspect this was simply a mistake, since the commit that introduced
it (
65e183af, “opcodes: Add opcode parameter definitions”) did lots of
bulk updates.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Thu, 15 Nov 2012 10:16:09 +0000 (11:16 +0100)]
Add opcode for running commands remotely
The opcode doesn't pay attention to the build-time flag to enable or
disable restricted commands. In a cluster different nodes could have
different settings.
Node locks are acquired in shared mode by default, but the use of an
exclusive lock can be forced using the “use_locking” parameter.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 13 Nov 2012 20:11:34 +0000 (21:11 +0100)]
Add RPC for restricted commands
Expose the functionality to run restricted commands remotely via RPC.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Fri, 16 Nov 2012 16:43:50 +0000 (17:43 +0100)]
locking: Don't schedule pending acq. for short timeout
Scheduling a pending acquisition is relatively expensive and lot of code
is involved. Unless there is already one, a new pipe needs to be opened.
Data structures need to be updated as well, only to be undone shortly
after. This patch adds a small condition to return straight away if the
timeout is shorter than 1ms and the lock couldn't be acquired right
away. Blocking acquisitions are not affected, as are acquisitions with
longer timeouts.
While I tried hard, I could not find a way of writing a test
specifically for this case which wouldn't have involved making
significant other modifications. Other tests already exercise the new
code, though.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michele Tartara [Mon, 19 Nov 2012 09:30:20 +0000 (10:30 +0100)]
Prevent hlint crash on unicode chars
Some versions of hlint crash on the Attoparsec test file because they
are not able to correctly handle unicode characters if the proper encoding
(--utf8) is not specified.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Fri, 16 Nov 2012 15:38:17 +0000 (16:38 +0100)]
Rename design document for SSH setup
More design details will be added regarding setting up the node daemon,
so this is more about adding a node than just setting up SSH.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 13 Nov 2012 19:07:55 +0000 (20:07 +0100)]
backend: Implement remote commands
As per design document (doc/design-remote-commands.rst), a number of
rather strict tests is applied to any incoming request, a delay is
inserted upon errors and returned error messages are very generic
(unless it's the actual command that failed). There are unit tests for
all of the newly added code.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 14 Nov 2012 10:23:33 +0000 (11:23 +0100)]
configure: Add option to enable remote commands
By default remote commands are disabled and need to be explicitely
enabled at build time.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Wed, 14 Nov 2012 01:46:01 +0000 (02:46 +0100)]
Add unit test for RAPI handler access definitions
- Ensure query-related resources have the same access permissions
(specifically “/2/query/*” and “/2/*/console”)
- Check access permission consistency (write implies read)
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Thu, 8 Nov 2012 18:35:42 +0000 (19:35 +0100)]
Update ganeti-rapi man page for new user option
Instead of duplicating what is already described in the full
documentation, a reference is added instead. Man pages can't use
constants or assertions, therefore it is easier for them to get out of
sync with the code.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Thu, 8 Nov 2012 18:31:14 +0000 (19:31 +0100)]
rapi: Add new user option for querying
This was requested in issue 301. Before this patch, requests to
“/2/query/*” and “/2/instances/*/console” would require authentication
with a user with write access. Since that is not strictly necessary, a
new user option named “read” is added.
Console information can also be retrieved as a normal query, therefore
the change applies there too.
This was the first user option to be added after “write”, therefore
quite a few changes were necessary. Documentation, including NEWS, is
updated as well.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michele Tartara [Fri, 16 Nov 2012 12:23:02 +0000 (13:23 +0100)]
Add Utility functions for loading data from test files
They mimic their python counterparts.
Added functions:
* getSourceDir
* testDataFilename
* readTestData
* readPythonTestData
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michele Tartara [Fri, 16 Nov 2012 08:53:37 +0000 (09:53 +0100)]
Improved TestHelper module docstring
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Thu, 15 Nov 2012 09:01:38 +0000 (10:01 +0100)]
Move gnt_cluster.SHOW_MACHINE_OPT to cli
This allows the option to be re-used in other places.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Tue, 8 Nov 2011 12:13:07 +0000 (12:13 +0000)]
Add ganeti-node-role ocf example file
This allows offlining nodes that don't respond if they are part of a
linux-HA cluster.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Guido Trotter [Fri, 4 Nov 2011 12:46:25 +0000 (12:46 +0000)]
Add ganeti-master-role.ocf example file
This allows controlling the cluster master role if the nodes are part of
a linux-HA cluster.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Iustin Pop [Tue, 13 Nov 2012 16:27:23 +0000 (17:27 +0100)]
Cleanup THH function use from built module namespace
Currently, THH.hs "injects" into the built code names of library
functions like Text.JSON.makeObj, Ganeti.JSON.fromObj, etc. built
directly from strings, via (e.g.)
varE (mkName "makeObj")
This means that the "makeObj" name must exist in the target module,
i.o.w. must be imported there. This leads to the strange case of
having to have imports that do not appear at all in the used
(template) code, but are needed to satisfy this "hidden" dependency;
look at Ganeti/Jobs.hs before this patch, for example.
This is also not very obvious, because we usually import Text.JSON
anyway; I only stumbled upon it while doing some cleanup work.
So to clean this up, the current patch changes the THH.hs to use not
string-derived, but identifier-derived names («'identifier» versus
«mkName "identifier"»); this is better, as the names must be
resolvable when compiling THH itself (once), and not when compiling
the multiple derived modules. As you can see, this allows removal of
extraneous imports from various modules.
Background information: an `mkName "foo"` results in a name of flavour
NameS (“An unqualified name; dynamically bound”) or alternatively to a
qualified name, but still dynamically bound. Whereas what we want is a
statically bound name: `'foo` results in a NameG flavour, “Global name
bound outside of the TH AST: An original name”.
One more explanation: the change is similar to going from 'x = eval
"map"' to 'x = map'; the name is no longer dynamically evaluated, but
statically when the module is compiled. In our case, previously names
were bound at target module compile time, now they are bound at THH.hs
compile time.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Thu, 15 Nov 2012 05:31:12 +0000 (06:31 +0100)]
Fix typo in manpage for gnt-os
s/exists/exist/
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Iustin Pop [Tue, 13 Nov 2012 15:20:34 +0000 (16:20 +0100)]
Switch Attoparsec/unicode test from QC to HUnit
This is not a quickcheck property, since it doesn't have any
variable/arbitrary inputs. So let's make it a test case, and shorten a
bit the name.
Also, sorry for not catching this in the review.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Michael Hanselmann [Wed, 14 Nov 2012 01:45:46 +0000 (02:45 +0100)]
Warn on invalid lines in HTTP user files
Without this change, invalid lines or values would be silently ignored.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 14 Nov 2012 08:39:53 +0000 (09:39 +0100)]
Don't check for remote command directory as file storage
This test does not work properly if localstatedir is not “/etc”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Guido Trotter [Wed, 14 Nov 2012 08:40:58 +0000 (09:40 +0100)]
gnt-instance.8: Add index to disk and nic removal
The text of the manpage explains that an index can be prepended to
"remove" but the short help doesn't mention it. Adding it helps making
the syntax clear.
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Michael Hanselmann [Wed, 14 Nov 2012 08:02:53 +0000 (09:02 +0100)]
Document master daemon's shutdown behaviour
This wasn't clearly documented until now.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Mon, 5 Nov 2012 17:49:23 +0000 (18:49 +0100)]
pathutils: Add directory for remote commands
Also add tests to ensure it's never allowed as a file storage path. A
constant for the lock file is also added.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 13 Nov 2012 19:11:10 +0000 (20:11 +0100)]
utils.process.RunResult: Always set "fail_reason" attribute
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 13 Nov 2012 19:10:39 +0000 (20:10 +0100)]
Add previously missing node daemon GID to getent mock
The UID is there, the GID wasn't.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 13 Nov 2012 19:10:32 +0000 (20:10 +0100)]
Add test utility to count calls to function
In some cases it's nice to verify a function has been called exactly N
times. This is going to be used in tests for remote commands.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Michael Hanselmann [Tue, 13 Nov 2012 20:18:37 +0000 (21:18 +0100)]
Sort content of generated RPC code
This is similar to commit f5ce761. A stable sort order makes changes
between versions easier to find.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Michael Hanselmann [Wed, 7 Nov 2012 16:39:50 +0000 (17:39 +0100)]
Add new test for RAPI
Unlike existing tests, this actually tests RAPI at the interface with
the HTTP server. This way authentification can also be tested. A test
for “/2/query/…” is included as it's a bit special.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Wed, 24 Oct 2012 01:46:15 +0000 (03:46 +0200)]
Add “gnt-job change-priority” sub-command
This can be used to change the priority of a pending or running job (the
latter only if there are unprocessed opcodes).
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Michael Hanselmann [Wed, 24 Oct 2012 01:46:01 +0000 (03:46 +0200)]
Expose changing job priority via LUXI
A new LUXI request is added, in both Python and Haskell.
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>