Statistics
| Branch: | Tag: | Revision:

root / lib / backend.py @ ff9efc03

History | View | Annotate | Download (68.4 kB)

# Date Author Comment
25361b9a 12/02/2008 04:35 pm Iustin Pop

Fix gnt-cluster verify w.r.t. rpc changes

This partially reorganizes the cluster verify LU:
- introduce constants for the node verify rpc call
- move from additional rpc calls to a single rpc call, the
call_node_info, which gaters all data needed...

89b14f05 12/02/2008 07:04 am Iustin Pop

Simplify a little the ssconf update

We have (again) the KeyToFilename function, so we move the writing of
the files to a method under SimpleStore.

Reviewed-by: imsnah

93384844 12/02/2008 07:03 am Iustin Pop

Revert "Get rid of ssconf"

This partially reverts the "Get rid of ssconf" patch.

It adds back a simpler version of the SimpleStore class, and drops the
WritableSimpleStore class. The new version of the class also has
node_list as a new key, and increases the size of the keys so that big...

726d7d68 11/27/2008 05:13 am Iustin Pop

Fix gnt-backup export

This patch fixes a bug in disk calculation for gnt-backup export, which
completely broke one-disk instance export.

The patch also corrects some error messages and style issues.

Reviewed-by: ultrotter

03d1dba2 11/25/2008 02:37 pm Michael Hanselmann

Pass ssconf values from master to node

Instead of parsing the configuration on the node, we pass the ssconf
values from the master.

Reviewed-by: iustinp

955db481 11/24/2008 11:46 am Guido Trotter

Correct GetAllInstancesInfo rtype

GetAllInstancesInfo, in the backend, returns just a dict, not a dict of dicts.

Reviewed-by: iustinp

6ddc95ec 11/21/2008 12:42 pm Michael Hanselmann

Add RPC call to update ssconf files

Reviewed-by: iustinp

d868edb4 11/11/2008 12:58 pm Iustin Pop

Convert trunk to posix-compatibility

We change two functions to use RunCmd without shell, and the other
(which needs a ssh command line) is changed to the '>... 2>&1' syntax.

Reviewed-by: imsnah

10c2650b 10/25/2008 03:37 am Iustin Pop

Update backend.py docstrings

This patch converts all of backend.py to epydoc formatting.

Reviewed-by: imsnah

3f5bd234 10/25/2008 03:37 am Iustin Pop

Fix another error handling case

The return from this error path is a dict, but the actual return value
(on the non-error handling) is a list of dicts. Change accordingly.

Reviewed-by: imsnah

0a70a72a 10/24/2008 07:14 pm Iustin Pop

Fix an error handling case

Found while reviewing documentation.

Reviewed-by: ultrotter

2e3d1946 10/24/2008 06:23 pm Guido Trotter

OSFromDisk remove superfluous empty line

Reviewed-by: iustinp

b1206984 10/24/2008 05:44 pm Iustin Pop

Some documentation updates

This fixes a few doc issues and converts a few docstrings to epydoc.

Reviewed-by: imsnah

74c47259 10/23/2008 05:19 pm Iustin Pop

Export the disk index in the import/export scripts

We want to export the disk index as some OSes will only want to export
the first disk (or the second one, etc.), even if we have multiple
disks.

The patch also updates the backend.ExportSnapshot docstring....

6c0af70e 10/22/2008 05:09 pm Guido Trotter

Convert ImportOSIntoInstance to OS API 10

- Change ImportOSIntoInstance not to get any "os_disk" and "swap_disk"
arguments but to accept multiple target images to import, and to
return a list of booleans with the result of each import
- Change the relevant rpc call and the only caller to conform...

0607699d 10/22/2008 05:09 pm Guido Trotter

Convert ExportSnapshot to OS API 10

We pass the data via the environment rather than on the command line, as
API 10 says. All the rest remains the same, and we export just one disk,
as the master calls this function for every snapshotted disk.

Reviewed-by: iustinp

19d7f90a 10/22/2008 05:08 pm Guido Trotter

LUExportInstance: snapshot all disks

Rather than just snapshotting the "sda" disk, we'll snapshot all of the
instance disks. If we can't snapshot a disk for any reason we'll log an
error and proceed anyway: in this case the resulting export will miss a
disk. This also changes all the warning messages to self.LogWarning()....

e9e9263d 10/22/2008 05:08 pm Guido Trotter

Convert SnapshotBlockDevice's docstring to epydoc

Reviewed-by: iustinp

d15a9ad3 10/17/2008 05:37 pm Guido Trotter

Cleanup os_add/rename rpc for OS API 10

- remove now unused osdev and swapdev arguments from backend, noded,
rpc, cmdlib
- convert docstrings to epydoc

Reviewed-by: iustinp

d324e3fc 10/17/2008 05:36 pm Guido Trotter

Temporarily explicitely break import/export

Since they're not converted to API 10 yet, we temporarily disable the
import/export functions.

Reviewed-by: iustinp

58f6e5ca 10/17/2008 05:36 pm Guido Trotter

AddOSToInstance: convert to api10

Reviewed-by: iustinp

ff38b6c0 10/17/2008 05:36 pm Guido Trotter

RunRenameInstance: convert to api10

Reviewed-by: iustinp

2266edb2 10/17/2008 05:36 pm Guido Trotter

Add new OSEnvironment function

This function calculates the basic environment for OS scripts in API
version 10.

Reviewed-by: iustinp

62dbbe7e 10/17/2008 05:35 pm Guido Trotter

OSFromDisk: use script names from constants

Reviewed-by: iustinp

8ee4dc80 10/17/2008 05:35 pm Guido Trotter

Change OSFromDisk's docstring to epydoc

Plus update it with the real variable name

Reviewed-by: iustinp

5d672980 10/15/2008 04:13 pm Iustin Pop

Add a rpc call for changing the drain flag

A new multi-node call is added that sets/resets the drain flag.

Reviewed-by: imsnah

51de46bf 10/14/2008 02:29 pm Iustin Pop

Change the backend to use the beparams

The backend.FinalizeExport function is changed to use the beparams
instead of the instance attributes. Future enhancements should be done
in order to export and import/reuse the whole be/hv params.

Reviewed-by: ultrotter

f23b5ae8 10/14/2008 01:20 pm Iustin Pop

Temporary fix for dual hvm/pvm instances

We have a problem with the current model of combining instance lists
from multiple hypervisors: we don't allow duplicates, but "xm list"
gives the same output for both pvm and hvm. This is a lack in the actual
xen hypervisor implementation/split between pvm and hvm, but for now we...

6217e295 10/14/2008 01:19 pm Iustin Pop

Export the hypervisor.ValidateParameters over RPC

The newly-added node-specific ValidateParams hypervisor method is
exported over RPC, using the semi-standard (success, message) return
value. Multi-node call, so that we call on both primary and secondary at...

caad16e2 10/12/2008 11:40 pm Iustin Pop

Abstract checking own address into a function

Currently, we check if we have a given ip address (i.e. it's alive on
one of our interfaces) but manually calling TcpPing(source=localhost).
This works, but having it spread all over the code makes it hard to...

082a7f91 10/10/2008 07:04 pm Guido Trotter

OS API: support for multiple versions in an OS

Allow multiple api versions in an OS. This is according to the OS API
changes design doc, by which an OS can support multiple versions of the
Ganeti API and if one is supported by Ganeti it will work. Since up to...

e69d05fd 10/08/2008 01:36 pm Iustin Pop

Move the hypervisor attribute to the instances

This (big) patch moves the hypervisor type from the cluster to the
instance level; the cluster attribute remains as the default hypervisor,
and will be renamed accordingly in a next patch. The cluster also gains...

9f0e6b37 10/07/2008 02:39 pm Iustin Pop

rpc.call_instance_migrate: pass the whole instance

Currently the call_instance_migrate call only passes the instance name;
we need to pass the whole object for the hypervisor_type changes (all
the other individual instance rpc calls already pass the instance...

62c9ec92 10/06/2008 06:58 pm Iustin Pop

backend.py change to get cluster name from master

Currently there are three function in backend that need the cluster name
in order to instantiate an SshRunner. The patch changes these to get the
cluster name from the master in the rpc call; once the multi-hypervisor...

6b0469d2 10/06/2008 04:16 pm Iustin Pop

Fix SshRunner breakage from the changed API

More places actually use the SshRunner than just the gnt-cluster
commands.

Reviewed-by: ultrotter

7688d0d3 10/01/2008 08:35 pm Michael Hanselmann

Convert ssh.py

Get rid of ssconf and convert to configuration instead.

Reviewed-by: iustinp

3707f851 10/01/2008 08:34 pm Michael Hanselmann

Convert hypervisor

Replacing ssconf with configuration.

Reviewed-by: iustinp

c657dcc9 10/01/2008 08:34 pm Michael Hanselmann

Convert backend.py

Replacing ssconf with simpleconfig.

Reviewed-by: iustinp

1bc59f76 09/09/2008 03:47 pm Michael Hanselmann

Never remove job queue lock in node daemon

Otherwise, corruption could occur in some corner cases. E.g. when
LeaveNode is running in a child and is in the process of removing
queue files, the main process gets killed, started again and gets
a request to update the queue. This is rather extreme corner case,...

bd1e4562 09/09/2008 03:24 pm Iustin Pop

Change backend._GetMasterInfo to return more data

The _GetMasterInfo() function needs to export the master name too to be
useful in master safety checks. This patch makes it a public (no _)
function and adds a third element in the return tuple. Its callers are...

4f0afaf5 08/14/2008 01:27 pm Guido Trotter

Pass hypervisor type to the OS scripts

It's handy to make the os scripts know which hypervisor the instance is
going to run under. In order not to change the os API we pass this
information in the environment, where the os scripts can access it if
they're hypervisor-aware....

24fc781f 08/08/2008 02:23 pm Michael Hanselmann

Don't always remove queue lock when queue is purged

The lock should only be removed if ganeti-noded is going to quit.
Otherwise it needs to be kept to prevent another process from creating
it again while we're still holding the (removed) lock. This is due to...

76ab5558 08/08/2008 02:22 pm Michael Hanselmann

backend: Add optional exclusion list to _CleanDirectory

The code cleaning the queue will make use of it.

Reviewed-by: iustinp

af5ebcb1 08/08/2008 02:21 pm Michael Hanselmann

noded: Add RPC function to rename job queue files

This will be used to archive jobs.

Reviewed-by: iustinp

dc31eae3 08/08/2008 02:21 pm Michael Hanselmann

backend: Add function to check whether file is in queue dir

Another function will need to check whether its parameters
are job queue files.

Reviewed-by: iustinp

553f1c1d 08/08/2008 01:02 pm Michael Hanselmann

Disallow uploading job queue files through upload_file

The job queue is now updated through its own RPC functions.

Reviewed-by: iustinp

ca52cdeb 08/08/2008 01:01 pm Michael Hanselmann

Add job queue RPC functions

jobqueue_update: Uploads a job queue file's content to a node. The
most common operation is to upload something that we already have
in a string. Unlike in the upload_file function, the file is not
read again when distributing changes, but content has to be passed...

3956cee1 08/08/2008 01:00 pm Michael Hanselmann

Move function cleaning directory to module level

JobQueuePurge() will be used by an RPC function.

Reviewed-by: iustinp

f78346f5 08/06/2008 02:27 pm Michael Hanselmann

Clean job queue directories when leaving cluster

Old job files shouldn't be left on nodes removed from a cluster.

Reviewed-by: iustinp

afee8008 07/31/2008 05:42 pm Michael Hanselmann

Allow job queue files to be uploaded through ganeti-noded

This is needed for job queue replication.

Reviewed-by: iustinp

38206f3c 07/30/2008 05:04 pm Iustin Pop

Fix pylint-detected issues

This is mostly:
- whitespace fix (space at EOL in some files, not all, broken
indentation, etc)
- variable names overriding others (one is a real bug in there)
- too-long-lines
- cleanup of most unused imports (not all)...

3b9e6a30 07/30/2008 04:27 pm Iustin Pop

Fix some errors detected by pylint

Reviewed-by: imsnah

b1b6ea87 07/30/2008 11:43 am Iustin Pop

Rework master startup/shutdown/failover

This (big) patch reworks the master startup/shutdown and the fixes the
master failover.

What does the patch do?

For master start/stop:
- remove the old ganeti-master script and its associated man page
- moves the ip start/stop directly into the backend.(Start|Stop)Master...

1c65840b 07/30/2008 11:32 am Iustin Pop

Add a new parameter to backend.(Start|Stop)Master

This patch adds a new, unused for now, parameter to the start and stop
master operations in backend. The idea behind it is that we need to be
able to control whether the IP (de)activation is coupled with daemon...

c3f0a12f 07/23/2008 01:06 pm Iustin Pop

Distribute the queue serial file after each update

This patch adds distribution of the queue serial file after each write
to it (but before a new job is created and written with that ID, and
before a response is returned, so we should be safe from crashes in...

18682bca 07/11/2008 06:45 pm Iustin Pop

Convert backend.py to the logging module

The patch also switches some of the exception logs to use
logging.exception (and therefore the log message will have a diferent
format).

(Note that this might not be a good choice in all cases, though)

Reviewed-by: imsnah

a17a7623 07/11/2008 04:54 pm Iustin Pop

Fix backend.NodeVolumes handling of LVM output

This is the same fix as for GetVolumeList.

I've checked manually and all other places that call lvm commands are
already checking the output validity in terms of correct number of
fields.

Reviewed-by: ultrotter

df4c2628 07/11/2008 04:23 pm Iustin Pop

Fix backend.GetVolumeList handling of LVM output

Sometimes ‘lvs’ can spit error messages on stdout, even when one wants
to parse the output:
...
Inconsistent metadata copies found - updating to use version 2776
...

So we need to validate the output to guard against such cases....

90fae627 06/27/2008 05:27 pm Guido Trotter

Allow VNC_PASSWORD_FILE to be rpc-uploaded

What could possibly go wrong?

Reviewed-by: iustinp

6d8b6238 06/27/2008 05:27 pm Guido Trotter

raise QuitGanetiException in LeaveCluster

Reviewed-by: iustinp

d61cbe76 06/20/2008 02:04 pm Iustin Pop

Add a rpc call for BlockDev.Close()

This patch adds rpc layer calls (in rpc.py and the equivalent in
ganeti-noded) to close a list of block devices, and the wrapper in
backend.py that takes a list of Disk objects, identifies them and
returns correctly formatted results....

594609c0 06/16/2008 07:06 pm Iustin Pop

Expose block device grow in backend.py

This patch adds a wrapper over the block device grow operation that
converts the input and output parameters as needed for the rpc layer.

Reviewed-by: imsnah

2a10865c 06/16/2008 04:57 pm Iustin Pop

Add migration support at the rpc layer

This patch adds the migration rpc call and its implementation in the
backend. The patch does not deal with the correct activation of disks.

Because of the new RPC, the protocol version is increased.

Reviewed-by: imsnah

9d4bfc96 05/13/2008 05:33 pm Iustin Pop

Implement node daemon conectivity tests

This patch adds in gnt-cluster verify checks for inter-node tcp
communication checks on the node daemon port for both the primary and
(if defined) secondary networks.

The output looks like (4-node cluster, one with the secondary interface...

b544cfe0 05/13/2008 10:32 am Iustin Pop

Reduce chance of ssh failures in verify cluster

The cluster verify builds a sorted list of nodes and passes that to all
the nodes (in parallel) for ssh checks. This means that for a cluster
with N nodes, there will be approximately N simultaneous connections to...

7b7857ee 04/30/2008 12:29 pm Guido Trotter

Remove non-existing arguments from some docstrings

A fewdocstrings in the HooksRunner backend class list arguments the relevant
functions do not take. Clean them up.

Reviewed-by: iustinp

8d528b7c 04/28/2008 12:47 pm Iustin Pop

Move iallocator script execution to ganeti-noded

Currently the iallocator execution takes place in the master, which is a
violation of the current architecture, and will create problems with a
threaded master daemon.

This patch moves the execution to the backend, similar to the hooks...

66f93869 04/24/2008 06:56 pm Manuel Franceschini

backend.FinalizeExport: safely initialize some vars

This patch initializes nic_count and disk_count with 0. This prevents
some reference errors if the snap_disks block device list is empty.

Reviewed-by: ultrotter

57c177af 04/10/2008 04:41 pm Iustin Pop

Move the OS search code into an abstract function

Based on the previous OS search code changes, we can now move the OS
search code into a generic look-for-file function in utils.py. This
means that the allocator code can use the same function.

Reviewed-by: ultrotter

c34c0cfd 04/10/2008 04:40 pm Iustin Pop

Change backend._OSSearch return values

Currently, the function backend._OSSearch() returns the (first) base dir
in which this OS can be found. Thereafter the full actual path to the OS
dir is built in the backend.OSFromDisk() function.

This patch changes this so that _OSSearch() always returns the full path...

778b75bb 04/05/2008 03:07 pm Manuel Franceschini

Backend directory functions for file backend

Add _[Create,Remove,Rename]FileStorageDir function which are needed for
file-based instance management. These function check whether the given
directory to operate on is under the cluster-wide defined default file...

c92b310a 03/18/2008 03:02 pm Michael Hanselmann

Move SSH functions into a class

This renames some functions and does some minor codestyle cleanup.

Reviewed-by: ultrotter

41a57aab 03/05/2008 06:48 pm Michael Hanselmann

Replace custom file writing code with utils.WriteFile

Reviewed-by: ultrotter

c9064964 02/29/2008 06:32 pm Iustin Pop

Fix master role stop on cluster destroy

Currently the cluster destroy doesn't remove the master role, which
means that the IP address of the cluster remains assigned to the master
node.

This patch fixes this and also a docstring in backend.StopMaster()....

6c8af3d0 02/22/2008 06:12 pm Manuel Franceschini

Small comment fix.

c99a3cc0 02/22/2008 06:12 pm Manuel Franceschini

Fixes small spell mistakes and comments

40a03283 02/14/2008 05:54 pm Iustin Pop

Alter the device activation code

This tiny patch fixes the breakage that the previous patch about
activation did by removing the Close() call after activation.

The initial reason for that call was that if the device is already
active and open, but we need it closed, we close it automatically....

1cafd236 01/30/2008 01:14 pm Guido Trotter

Export bridge information too

gnt-backup export used to export the ip and mac of each nic, but not which
bridge it was connected to. Adding this information.

Reviewed-by: iustinp

d87ae7d2 01/21/2008 04:33 pm Iustin Pop

Fix VG listing broken by r510

LVM code sometimes adds an extra separator at the end of the field list.
Make the code strip it if exists.

Reviewed-by: imsnah

f4d377e7 01/21/2008 12:13 am Iustin Pop

Make backend._GetVGInfo check the validity of 'vgs'

Currently, the function backend._GetVGInfo only checks for errors via
the exit code of the 'vgs' command. However, there are other ways of
failure so we need to also check for valid output before parsing....

97628462 01/21/2008 12:09 am Iustin Pop

Change a hardcoded path into its proper constant

The function backend.UploadFile still uses "/etc/hosts" directly instead
of the existing constant; this patch fixes this.

Reviewed-by: ultrotter

147af04d 01/16/2008 06:01 pm Iustin Pop

Two small style fixes

This is a merge from the 1.2 branch

Reviewed-by: imsnah

b63ed789 01/07/2008 03:30 pm Iustin Pop

Improve verify-disks: broken/missing LV detection

This patch improves the ‘gnt-cluster verify-disks’ command by adding
support for detecting broken volume groups and missing logical volume
names.

As such, we don't try anymore to activate disks for instances that are...

cb2037a2 12/11/2007 06:25 pm Iustin Pop

Return more data in rpc.call_volume_list

Currently, the volume_list call returns only the volume size. However,
it is useful to also have two other things: the 'inactive' state of the
volume (which might trigger a ‘vgchange -a y’ on the volume group) and...

ff73280e 12/04/2007 01:21 pm Michael Hanselmann

On OS creation errors, write logfile path to ganeti-noded's logfile.

Reviewed-by: schreiberal

f675d4aa 12/04/2007 01:18 pm Alexander Schreiber

Output reading fix for backend.NodeVolumes()

Use result.stdout instead of result.output to avoid potential confusion
by merged in error messages from stderr.

Reviewed-by: iustinp

f49a5e02 12/04/2007 12:37 pm Alexander Schreiber

Modify GetVolumeList so output on stderr from lvs doesn't break it.

Reviewed-by: imsnah

f4bc1f2c 12/03/2007 04:03 pm Michael Hanselmann

Various code style fixes for strings.

- When line wrapping is needed, move spaces to the next line.
- Remove embedded line breaks from error messages.

Reviewed-by: schreiberal

20a0c9ef 11/14/2007 11:52 pm Guido Trotter

When an assembly error occurs log it too

Right now an assembly error produces an exception but not a log message. This
is bad because the exception suggests looking at the log, but the log itself
has a lot of errors which are not really a problem and only some which really...

7803d4d3 11/13/2007 01:11 am Iustin Pop

Fix a wrong comparison in _RecursiveAssembleBD

We want to prevent sending too many 'None' children to a device.
However, the test as it is today is wrong, as we want to test the
situation after adding a new child, and not before. This patch fixes
this by testing greater-or-equal instead of just greater....

c8a0948f 11/09/2007 04:54 pm Michael Hanselmann

Use new functions to modify /etc/hosts.

Reviewed-by: schreiberal

0834c866 11/07/2007 05:53 pm Iustin Pop

Enhance secondary node replace for drbd8

This (big) patch does two things:
- add "local disk status" to the block device checks
(BlockDevice.GetSyncStatus and the rpc calls that call this
function, and therefore cmdlib._CheckDiskConsistency)
- improve the drbd8 secondary replace operation using the above...

fc1dc9d7 11/06/2007 07:00 pm Iustin Pop

Allow DRBD8 operation without backing storage

This patch adds the following functionality:
- DRBD8 devices can assemble without local storage (done by allowing
None in the list of children, and making DRBD8 to ignore all
children if any is None)...

e739bd57 11/06/2007 07:00 pm Iustin Pop

Change the way remove children is called in bdev

For some cases, we don't have to have access to the children of a device
in order to remove them (e.g. md over lvs, or drbd over lvs). In order
to ease the removal process, skip over finding the child if it provides...

cf5a8306 11/05/2007 05:08 pm Iustin Pop

Fix a unhandled error case in device creation

The block device creation process is the following:
- device create
- device assembly (on primary or depending on dev_type, on secondary
too)
- set sync speed
- return

The problem is that device assembly after creation was not checked for...

65fe4693 11/05/2007 05:06 pm Iustin Pop

Miscellaneous style fixes

This patch fixes some minor pylint warnings (unused variables, wrong
indentation, etc.) and a real bug in the recovery for drbd8 rename
procedure.

Reviewed-by: imsnah

8fa42c7c 11/05/2007 02:43 am Guido Trotter

Make DiagnoseOS use the modified OS objects

Modify backend.py so that DiagnoseOS only returns OS objects rather than
InvalidOS errors, and make sure gnt-os understands the new objects. Also delete
the deprecated helper functions from gnt-os.

Reviewed-By: iustinp

74b4217d 11/04/2007 05:37 am Guido Trotter

Fix two typos in a doc string

Remove a wrong "i" and add a missing ")" to the DiagnoseOS function doc string.

Reviewed-By: iustinp

3f78eef2 11/02/2007 03:44 pm Iustin Pop

Implement device to instance mapping cache

Currently, troubleshooting DRBD problems involves a manual process of going
backwards from the DRBD device to the instance that owns it.

This patch adds a weak (i.e. not guaranteed to be correct or up-to-date)
cache of device to instance. The cache should be, in normal operation,...

5d0fe286 11/01/2007 07:22 pm Iustin Pop

Whitespace fixes

Reviewed-by: ultrotter