History | View | Annotate | Download (40.1 kB)
Add a few more checks to verify config
- Check that the enabled hypervisors list is valid- Check that the master node is a valid node
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Remove ConfigWriter.InitConfig
It's been replaced by a simpler bootstrap.InitConfig function, whichdoes the same job, and is currently unused.
Fix pylint warnings
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix some typos
Allow GetMasterCandidateStats to ignore some nodes
This patch modifies ConfigWriter.GetMasterCandidateStats to allow it toignore some nodes in the calculation, so that we can use it to predictcluster state without some nodes (which we know we will modify, and thus...
Add a new ssconf file with the cluster tags
Since the cluster tags are/should be more-or-less static, add them as anssconf key, so that querying them is possible without creating ajob/requiring the masterd to be running.
Reviewed-by: imsnah
Fix some epydoc style issues
99% of the epydoc return tags are "@return:", but each of the modified fileshad one "@returns:" line. We fix this for consistency.
ConfigWriter.AddInstance check instance mac
There is a race condition in CreateInstance, since the mac address isgenerated early and only added to the config (and thus really assured tobe unique) only at this point. Since it's possible that another instance...
Do not check 'None' disk IDs for duplicates
In case of 'None' logical or physical IDs, we don't need to check themfor duplicates. This case can happen for DRBD devices in case of newlyadded disks, for example.
Prevent race condition on MAC addresses
This patch adds a temporary set for MACs that have been requested butare not yet in the configuration (as part of an instance NIC). The MACsof an instance are automatically removed from this set when the instance...
ConfigWriter: add checks for duplicate disk IDs
This patch adds a safety check for duplicate disk logical/physical IDs,in order to prevent possible software bugs.
ConfigWriter: handle the drained node flag
This patch changes the master candidate pool computations inConfigWriter to properly handle drained nodes. They are now excludedfrom counting towards the reachable number of candidates.
The patch also adds verification of consistency for the node status....
ssconf: add some more keys and some fixes
This patch adds the online node list and instance list to the ssconfkeys. In order to do distribute correctly the instance list, we need toupdate the cluster serial number on instance additions and removals.
The patch also changes the permissions on the ssconf files to be 0444:...
Implement disk verify checks in config verify
This patch adds a simple check that the 'mode' attribute of top-level disks iscorrect. It does not recurse over children.
The framework could be extended with other checks in the future.
Relax the restrictions on temporary DRBD minors
Currently the restrictions are too harsh: there is a time intervalbetween an instance gets a new disk and before it is added to theconfiguration in which the restriction is not met. We solve this byallowing temporary DRBD minors to match existing minors (for the same...
Introduce more configuration consistency checks
This patch enhances the duplicate DRBD minors checks (currently just afew) and adds automatic checks of configuration consistency atconfiguration file writing time.
In order to do so and show meaningful error messages, the...
Automatically release DRBD minors on success
This patch converts the DRBD minors reservation protocol from explicitrelease to automatic release on the success paths. On the errors paths,it's still needed to manual release.
The patch doesn't bring much by itself, but is needed for a future patch...
Change the instance status attribute to boolean
Due to historic reasons, the “should run or not” attribute of aninstance was denoted by its “status” attribute having a string value ofeither ‘up’ or ‘down’. Checking this is in code was done via hardcoding...
Fix adding of disks to an instance
The ConfigWriter.AllocateDRBDMinor requires the instance name, not theinstance object. The LUSetInstanceParms is passing wrongly the instanceobject, which can cause breakage.
The patch also adds asserts to check for this mismatch in ConfigWriter....
Make cluster-verify check the drbd minors space
This patch adds support for verification of drbd minors space in clusterverify: minors which belong to running instances and should be onlinebut are not, and minors which do not belong to any instace but are in...
Add a new ssconf file with the ganeti version
The patch adds a new ssconf file containing the ganeti version.
Fix gnt-os for offline nodes
We shouldn't query offline nodes in gnt-os. This patch adds an utilityfunction to ConfigWriter that returns the names of online nodes and usesit in LUDiagnoseOS to query only the good nodes.
cleanup: ConfigWriter, initialize all attributes
We should initialized the _last_cluster_serial in the constructor too (just tobe consistent).
Reviewed-by: amishchenko
Fix epydoc format warnings
This patch should fix all outstanding epydoc parsing errors; as such, weswitch epydoc into verbose mode so that any new errors will be visible.
Fix _AdjustCandidatePool
Currently the ConfigWriter.MaintainCandidatePool returns node names, and_AdjustCandidatePool uses them as such, but then it passes these tocontext.ReaddNode which in turn passes them to jqueue.JobQueue.AddNode whichuses them as objects.Node instances....
Add function to compute the master candidates
Since some nodes can be offline, we can't just take the length of thenode list as the maximum possible number of master candidates.
The patch adds an utility function to correctly compute this value andreplaces hardcoded computations with the use of this function. It then...
Add the offline node list to ssconf
The patch also changes the various node list generation to be moreconsistent.
A few fixes related to master candidates
This patch: - fixes cluster verify when all nodes are master candidates, but the candidate_pool_size is higher - warn when the master node is not marked as candidate - disable setting master node to regular node...
Add candidate pool size checks in verify
Add the list of master candidates to ssconf
Restrict config replication to master candidates
This patch restricts the config data replication to master candidatesonly.
Replicate the node list in ssconf
This patch adds node_list in the list of replicated values fromConfigWriter.
Fix logic bug in rev 2072
In revision 2072 "ConfigWriter: change cluster serial meaning" I misreadthe serial_no update logic: it was about updating the serial number onthe object itself, not on the cluster.
So we don't actually have at all cluster serial number increase when a...
ConfigWriter: change cluster serial meaning
Currently, we increase the cluster serial number for instance additions,removals and renames. This is conforming with the REST paradigm, howeverit means that for each of these operations, we need to push ssconf...
Pass ssconf values from master to node
Instead of parsing the configuration on the node, we pass the ssconfvalues from the master.
Reviewed-by: iustinp
Fix unittests broken by rev 2015
Ssconf files shouldn't be updated when running unittests.
Reviewed-by: ultrotter
Only update ssconf on cluster serial change
There is no need to update ssconf if the cluster serial number has notchanged.
Write ssconf files when updating configuration
Fix leftover of hypervisor attribute rename
Convert rpc.call_upload_file to use addresses
This patch allows rpc.call_upload_file to use addresses (if passed), andalso converts the ConfigWriter._DistributeConfig to pass them, sincethis is an often-done operation.
Convert rpc module to RpcRunner
This big patch changes the call model used in internode-rpc fromstandalong function calls in the rpc module to via a RpcRunner class,that holds all the methods. This can be used in the future to enablesmarter processing in the RPC layer itself (some quick examples are not...
Disable re-reading of config file
Since the objects read from the config file are passed to the variousthreads, it's unsafe to re-read the config file (and throw awayConfigWriter._config_data). As such, we disable the re-reading of thefile (since now the master is the owner the file, it makes not sense to...
Convert config.py
The configuration version is now again in the configuration file.
Add new query to get cluster config values
This can be used to retrieve certain cluster config values fromwithin clients.
OpDumpClusterConfig was not used anywhere, hence I'm just reusingit. The way ConfigWriter.DumpConfig returned the configurationwas not thread-safe, anyway (no deepcopy)....
Move functions from ssconf.py elsewhere
These functions will be used to access config values instead of usingssconf.
Add cluster options from ssconf to configuration
ssconf will become write-only from ganeti-masterd's point of view,therefore all settings in there need to go into the main configurationfile.
Move instantiation of config into bootstrap.py
Future patches will add even more variables to the cluster config.Adding more parameters wouldn't make the function easier to use andit doesn't make sense to pass them to another function, as it'sonly done once in bootstrap.py on cluster initialization....
Extend DRBD disks with shared secret attribute
This patch, which is similar to r1679 (Extend DRBD disks with minorsattribute), extends the logical and physical id of the DRBD disks with ashared secret attribute. This is generated at disk creation time and...
Fix a bug related to static minors
When the node does not yet have any minors allocated, the first minor(0) will not be entered in the ConfigWriter._temporary_drbds structure.This does not happen for our current usage, since we always ask for twominors (so the next call will not match this case), but it will be...
Add checks for tcp/udp port collisions
In case the config file is manually modified, or in case of bugs, thetcp/udp ports could be reused, which will create various problems(instances not able to start, or drbd disks not able to communicate).
This patch extends the ConfigWriter.VerifyConfig() method (which is used...
Update the cluster serial_no on certain operations
This patch adds update of the cluster serial number for: - add/remove node (as the cluster's node list is changed) - add/remove/rename instance (as the cluster's instance list is changed) - change the volume group name...
Initialize and update the serial_no on objects
This patch add initialization of the serial_no on instance and nodes,and update of the field whenever an object is updated in the genericcase, via ConfigWriter.Update(obj) and in the specific case ofinstances' state being modified manually....
Switch the global serial_no to the top object
Currently the serial_no that is incremented every time the configurationfile is written is located on the 'cluster' object in the configurationstructure. However, this is wrong as the cluster serial_no should be...
Switch config.py to logging
A couple of more modules are using the obsolete logger functions, configbeing one of them.
Implement config support for drbd static minors
This patch adds support for allocating static minors.
Like for the LVM uuids, we add a new cache for the temporarily allocatedrequests, and the users of the new methods must manually clear thecache. If this doesn't happen, at worst we lose some minors....
Extend DRBD disks with minors attribute
This patch converts the DRBD disks to contain also a minor (per eachnode) attribute. This minor is not yet used and is always initializedwith None, so the patch does not have any real-world impact - except forautomatically upgrading config files (it adds the minors as None, None)....
GetAllInstancesInfo, change internal iterator name
GetAllInstancesInfo used "node" as an iterator name. Change it toinstance to make it less confusing.
Add an atomic ConfigWrite.GetAllInstanceInfo()
In order to be able to query instance without locking them, we need thesame atomic query of multiple instances as for nodes.
Add ConfigWriter._UnlockedGetInstanceList/Info()
This patch splits the GetInstanceInfo and GetInstanceList methods intotwo parts, one locked one _Unlocked similar to the way nodes arequeried.
Add a way to export all node information at once
The patch adds a new function to export all node information at once(i.e. atomically with respect to the configuration lock).
Implement {Add,Readd,Remove}Node in GanetiContext
By doing this we've a central place which coordinates what needs to bedone when adding or removing nodes. Another patch will add calls intothe job queue.
Two log messages move to config.py.
When removing a node, node_leave_cluster is now called after it has...
Fix pylint-detected issues
This is mostly: - whitespace fix (space at EOL in some files, not all, broken indentation, etc) - variable names overriding others (one is a real bug in there) - too-long-lines - cleanup of most unused imports (not all)...
ConfigWriter: synchronize access
Since we share the ConfigWriter we need somehow to make sure thataccessing it is properly synchronized. We'll do it using thelocking.ssynchronized decorator and a module-private shared lock.
This patch also renames a few functions, which were called inside the...
ConfigWriter: remove _ReleaseLock
Remove empty function _ReleaseLock and all its calls. Since we onlyhave one configwriter per cluster the locking needs to cover all thedata in the object, and not just the file contents. Locking inConfigWriter will be handled using the ganeti locking library....
Use new ssconf function to check configuration version
Upgrades will be handled in future patches.
Wrap exception in _DistributeConfig code
nodelist.remove(X) could potentially raise a ValueError (even if the chancethat the current node is not in the list are pretty scarce, and its absenceshould raise a red flag anyway). If this happens let things go on, as that's...
Simplify target generation in DistributeConfig
Currently we get the list of nodes, and for each one extract all its info, andjust to exclude it if the name matches ours. Since the list of nodes is a listof names just use .remove() to exclude ourself from it, and use that list...
Only change instance status if not already so
This patch makes the _SetInstanceStatus only change the status of theinstance (in the config file) if it's not already in the desired state.This skips the increase of the config serial no without actual data...
Reuse common code between MarkInstance{Up,Down}
This patch creates a new method ConfigWriter._SetInstanceStatus thatconsists of the common code between MarkInstanceUp and MarkInstanceDown.These two methods become wrappers over the _SetInstanceStatus.
Abstract the json functions into a separate module
This simple patch adds a new module that holds the simplejson functionsfor serialization/deserialization. This reduces the amount of redundantcode.
The patch also adds some normalizations to the json output:...
Modify LURenameInstance to support file backend
This patch does two things:- Modify LURenameInstance.Exec to rename directory when a file-based instance is renamed- Modify config.RenameInstance() to replace the directory name in config.data for file devices...
Fix SetVGName() to access object not dict
Add method SetVGName() to config.py
Allow instance MAC address to be set.
Allow the MAC address of an instance to be specified optionally duringinstance creation and later to be changed via instance modify.
Fix logging of some checks in LUClusterVerify
Currently, the results of ConfigWriter.VerifyConfig() are lost. Makesure we log them too in cluster verify.
Rename utils.GetUUID to utils.NewUUID.
Reviewed-by: schreiberal
Initial implementation of drbd8 template type
This is a partially working drbd8 template type. It does: - add/remove - startup/failover/shutdown
Not working is replace disks, which needs custom code for this template.
Abstract more strings values into constants
Currently, the disk types are defined using constants in the code.Convert those into constants so that we can easily find them and checktheir usage.
Note that we don't rename the values of the constants as they are used...
Enable the ConfigWriter to keep track of updates
This allows external callers to determine if the configuration has beenchanged.
Use the kernel's ability to generate UUIDs.
This removes the dependency on either the uuid module or e2fsprogs' uuidgen.
Remove the shebang from modules
Since modules are not directly executables, remove the shebang fromthem. This helps with lintian warnings.
Also make the autogenerated _autoconf.py contain two comment lines atthe beginning, like the other modules.
Revert a debugging 'raise' from rev 195
As far as I understand, this was just a debugging aid. Remove it so thatconfiguration errors are nicely handled.
Change configuration storage format from Pickle to JSON.
- Add NEWS file with major changes between versions.- Bump RPC version number- No longer serialize in RPC, but just convert to dict
Old Pickle based configuration files can be converted using the cfgupgrade...
Remove requirement that host names are FQDN
We currently require that hostnames are FQDN not short names(node1.example.com instead of node1). We can allow short names as longas: - we always resolve the names as returned by socket.gethostname() - we rely on having a working resolver...
Add rename instance support to ConfigWriter
This patch adds support for renaming instances to the ConfigWriterclass. It's needed in here in order to guarantee that we don't lose theinstance during the rename (the instance will be there, either with the...
Style changes for pep-8 and python-3000 compliance.
This changes the raising of exceptions from: raise Exception, valueto raise Exception(value)
as the first form will be removed in python-3000 and the second form ispreferred now.
The changes also involve a few cases of changing from raising standard...
Prepare for the tags implementation: - add an Update method on the ConfigWriter class; - make the AddInstance method not log disks for diskless types (helps with testing) - implement limited testing for the ConfigWriter with focus on the new Update method
Re-stat the config file when we write it so that the next _OpenConfig()won't reread it if not modified by someone else.
Implement a standard way of getting the cluster object from theconfiguration.
Change logival volume names to not be based on the instance's name, butinstead use an UUID prefix and a suffix denoting the disk iv_name(sda/sdb) and possibly it's type (data/meta).
Move the cluster name from ConfigWriter to SimpleStore.
Reason: if left ConfigWriter, nodes don't know to which cluster they belong.This will bite us later when we'll revisit node join operation.
Cons: we lose the cluster name from the config file, which means a...
Don't pass master_node
Comment formatting updates.
The master_node is no longer stored in the config file (but in sstore)and all users have been converted, let's remove it from the slots of thecluster object.
Also let's move the tcpudp_port_pool to the cluster object.
Get ports from configuration if there are some.
Big change/cleanup in relation to the master startup: - move the master node name from the ConfigWriter to SimpleStore (all nodes need this, and it was the only thing pulled in from the ConfigWriter on nodes) - fix mcpu.py and the testing w.r.t. this change; for testing, rename...
Initial commit.