Statistics
| Branch: | Tag: | Revision:

root / lib / backend.py @ a1578d63

History | View | Annotate | Download (55.5 kB)

# Date Author Comment
1bc59f76 09/09/2008 03:47 pm Michael Hanselmann

Never remove job queue lock in node daemon

Otherwise, corruption could occur in some corner cases. E.g. when
LeaveNode is running in a child and is in the process of removing
queue files, the main process gets killed, started again and gets
a request to update the queue. This is rather extreme corner case,...

bd1e4562 09/09/2008 03:24 pm Iustin Pop

Change backend._GetMasterInfo to return more data

The _GetMasterInfo() function needs to export the master name too to be
useful in master safety checks. This patch makes it a public (no _)
function and adds a third element in the return tuple. Its callers are...

4f0afaf5 08/14/2008 01:27 pm Guido Trotter

Pass hypervisor type to the OS scripts

It's handy to make the os scripts know which hypervisor the instance is
going to run under. In order not to change the os API we pass this
information in the environment, where the os scripts can access it if
they're hypervisor-aware....

24fc781f 08/08/2008 02:23 pm Michael Hanselmann

Don't always remove queue lock when queue is purged

The lock should only be removed if ganeti-noded is going to quit.
Otherwise it needs to be kept to prevent another process from creating
it again while we're still holding the (removed) lock. This is due to...

76ab5558 08/08/2008 02:22 pm Michael Hanselmann

backend: Add optional exclusion list to _CleanDirectory

The code cleaning the queue will make use of it.

Reviewed-by: iustinp

af5ebcb1 08/08/2008 02:21 pm Michael Hanselmann

noded: Add RPC function to rename job queue files

This will be used to archive jobs.

Reviewed-by: iustinp

dc31eae3 08/08/2008 02:21 pm Michael Hanselmann

backend: Add function to check whether file is in queue dir

Another function will need to check whether its parameters
are job queue files.

Reviewed-by: iustinp

553f1c1d 08/08/2008 01:02 pm Michael Hanselmann

Disallow uploading job queue files through upload_file

The job queue is now updated through its own RPC functions.

Reviewed-by: iustinp

ca52cdeb 08/08/2008 01:01 pm Michael Hanselmann

Add job queue RPC functions

jobqueue_update: Uploads a job queue file's content to a node. The
most common operation is to upload something that we already have
in a string. Unlike in the upload_file function, the file is not
read again when distributing changes, but content has to be passed...

3956cee1 08/08/2008 01:00 pm Michael Hanselmann

Move function cleaning directory to module level

JobQueuePurge() will be used by an RPC function.

Reviewed-by: iustinp

f78346f5 08/06/2008 02:27 pm Michael Hanselmann

Clean job queue directories when leaving cluster

Old job files shouldn't be left on nodes removed from a cluster.

Reviewed-by: iustinp

afee8008 07/31/2008 05:42 pm Michael Hanselmann

Allow job queue files to be uploaded through ganeti-noded

This is needed for job queue replication.

Reviewed-by: iustinp

38206f3c 07/30/2008 05:04 pm Iustin Pop

Fix pylint-detected issues

This is mostly:
- whitespace fix (space at EOL in some files, not all, broken
indentation, etc)
- variable names overriding others (one is a real bug in there)
- too-long-lines
- cleanup of most unused imports (not all)...

3b9e6a30 07/30/2008 04:27 pm Iustin Pop

Fix some errors detected by pylint

Reviewed-by: imsnah

b1b6ea87 07/30/2008 11:43 am Iustin Pop

Rework master startup/shutdown/failover

This (big) patch reworks the master startup/shutdown and the fixes the
master failover.

What does the patch do?

For master start/stop:
- remove the old ganeti-master script and its associated man page
- moves the ip start/stop directly into the backend.(Start|Stop)Master...

1c65840b 07/30/2008 11:32 am Iustin Pop

Add a new parameter to backend.(Start|Stop)Master

This patch adds a new, unused for now, parameter to the start and stop
master operations in backend. The idea behind it is that we need to be
able to control whether the IP (de)activation is coupled with daemon...

c3f0a12f 07/23/2008 01:06 pm Iustin Pop

Distribute the queue serial file after each update

This patch adds distribution of the queue serial file after each write
to it (but before a new job is created and written with that ID, and
before a response is returned, so we should be safe from crashes in...

18682bca 07/11/2008 06:45 pm Iustin Pop

Convert backend.py to the logging module

The patch also switches some of the exception logs to use
logging.exception (and therefore the log message will have a diferent
format).

(Note that this might not be a good choice in all cases, though)

Reviewed-by: imsnah

a17a7623 07/11/2008 04:54 pm Iustin Pop

Fix backend.NodeVolumes handling of LVM output

This is the same fix as for GetVolumeList.

I've checked manually and all other places that call lvm commands are
already checking the output validity in terms of correct number of
fields.

Reviewed-by: ultrotter

df4c2628 07/11/2008 04:23 pm Iustin Pop

Fix backend.GetVolumeList handling of LVM output

Sometimes ‘lvs’ can spit error messages on stdout, even when one wants
to parse the output:
...
Inconsistent metadata copies found - updating to use version 2776
...

So we need to validate the output to guard against such cases....

90fae627 06/27/2008 05:27 pm Guido Trotter

Allow VNC_PASSWORD_FILE to be rpc-uploaded

What could possibly go wrong?

Reviewed-by: iustinp

6d8b6238 06/27/2008 05:27 pm Guido Trotter

raise QuitGanetiException in LeaveCluster

Reviewed-by: iustinp

d61cbe76 06/20/2008 02:04 pm Iustin Pop

Add a rpc call for BlockDev.Close()

This patch adds rpc layer calls (in rpc.py and the equivalent in
ganeti-noded) to close a list of block devices, and the wrapper in
backend.py that takes a list of Disk objects, identifies them and
returns correctly formatted results....

594609c0 06/16/2008 07:06 pm Iustin Pop

Expose block device grow in backend.py

This patch adds a wrapper over the block device grow operation that
converts the input and output parameters as needed for the rpc layer.

Reviewed-by: imsnah

2a10865c 06/16/2008 04:57 pm Iustin Pop

Add migration support at the rpc layer

This patch adds the migration rpc call and its implementation in the
backend. The patch does not deal with the correct activation of disks.

Because of the new RPC, the protocol version is increased.

Reviewed-by: imsnah

9d4bfc96 05/13/2008 05:33 pm Iustin Pop

Implement node daemon conectivity tests

This patch adds in gnt-cluster verify checks for inter-node tcp
communication checks on the node daemon port for both the primary and
(if defined) secondary networks.

The output looks like (4-node cluster, one with the secondary interface...

b544cfe0 05/13/2008 10:32 am Iustin Pop

Reduce chance of ssh failures in verify cluster

The cluster verify builds a sorted list of nodes and passes that to all
the nodes (in parallel) for ssh checks. This means that for a cluster
with N nodes, there will be approximately N simultaneous connections to...

7b7857ee 04/30/2008 12:29 pm Guido Trotter

Remove non-existing arguments from some docstrings

A fewdocstrings in the HooksRunner backend class list arguments the relevant
functions do not take. Clean them up.

Reviewed-by: iustinp

8d528b7c 04/28/2008 12:47 pm Iustin Pop

Move iallocator script execution to ganeti-noded

Currently the iallocator execution takes place in the master, which is a
violation of the current architecture, and will create problems with a
threaded master daemon.

This patch moves the execution to the backend, similar to the hooks...

66f93869 04/24/2008 06:56 pm Manuel Franceschini

backend.FinalizeExport: safely initialize some vars

This patch initializes nic_count and disk_count with 0. This prevents
some reference errors if the snap_disks block device list is empty.

Reviewed-by: ultrotter

57c177af 04/10/2008 04:41 pm Iustin Pop

Move the OS search code into an abstract function

Based on the previous OS search code changes, we can now move the OS
search code into a generic look-for-file function in utils.py. This
means that the allocator code can use the same function.

Reviewed-by: ultrotter

c34c0cfd 04/10/2008 04:40 pm Iustin Pop

Change backend._OSSearch return values

Currently, the function backend._OSSearch() returns the (first) base dir
in which this OS can be found. Thereafter the full actual path to the OS
dir is built in the backend.OSFromDisk() function.

This patch changes this so that _OSSearch() always returns the full path...

778b75bb 04/05/2008 03:07 pm Manuel Franceschini

Backend directory functions for file backend

Add _[Create,Remove,Rename]FileStorageDir function which are needed for
file-based instance management. These function check whether the given
directory to operate on is under the cluster-wide defined default file...

c92b310a 03/18/2008 03:02 pm Michael Hanselmann

Move SSH functions into a class

This renames some functions and does some minor codestyle cleanup.

Reviewed-by: ultrotter

41a57aab 03/05/2008 06:48 pm Michael Hanselmann

Replace custom file writing code with utils.WriteFile

Reviewed-by: ultrotter

c9064964 02/29/2008 06:32 pm Iustin Pop

Fix master role stop on cluster destroy

Currently the cluster destroy doesn't remove the master role, which
means that the IP address of the cluster remains assigned to the master
node.

This patch fixes this and also a docstring in backend.StopMaster()....

6c8af3d0 02/22/2008 06:12 pm Manuel Franceschini

Small comment fix.

c99a3cc0 02/22/2008 06:12 pm Manuel Franceschini

Fixes small spell mistakes and comments

40a03283 02/14/2008 05:54 pm Iustin Pop

Alter the device activation code

This tiny patch fixes the breakage that the previous patch about
activation did by removing the Close() call after activation.

The initial reason for that call was that if the device is already
active and open, but we need it closed, we close it automatically....

1cafd236 01/30/2008 01:14 pm Guido Trotter

Export bridge information too

gnt-backup export used to export the ip and mac of each nic, but not which
bridge it was connected to. Adding this information.

Reviewed-by: iustinp

d87ae7d2 01/21/2008 04:33 pm Iustin Pop

Fix VG listing broken by r510

LVM code sometimes adds an extra separator at the end of the field list.
Make the code strip it if exists.

Reviewed-by: imsnah

f4d377e7 01/21/2008 12:13 am Iustin Pop

Make backend._GetVGInfo check the validity of 'vgs'

Currently, the function backend._GetVGInfo only checks for errors via
the exit code of the 'vgs' command. However, there are other ways of
failure so we need to also check for valid output before parsing....

97628462 01/21/2008 12:09 am Iustin Pop

Change a hardcoded path into its proper constant

The function backend.UploadFile still uses "/etc/hosts" directly instead
of the existing constant; this patch fixes this.

Reviewed-by: ultrotter

147af04d 01/16/2008 06:01 pm Iustin Pop

Two small style fixes

This is a merge from the 1.2 branch

Reviewed-by: imsnah

b63ed789 01/07/2008 03:30 pm Iustin Pop

Improve verify-disks: broken/missing LV detection

This patch improves the ‘gnt-cluster verify-disks’ command by adding
support for detecting broken volume groups and missing logical volume
names.

As such, we don't try anymore to activate disks for instances that are...

cb2037a2 12/11/2007 06:25 pm Iustin Pop

Return more data in rpc.call_volume_list

Currently, the volume_list call returns only the volume size. However,
it is useful to also have two other things: the 'inactive' state of the
volume (which might trigger a ‘vgchange -a y’ on the volume group) and...

ff73280e 12/04/2007 01:21 pm Michael Hanselmann

On OS creation errors, write logfile path to ganeti-noded's logfile.

Reviewed-by: schreiberal

f675d4aa 12/04/2007 01:18 pm Alexander Schreiber

Output reading fix for backend.NodeVolumes()

Use result.stdout instead of result.output to avoid potential confusion
by merged in error messages from stderr.

Reviewed-by: iustinp

f49a5e02 12/04/2007 12:37 pm Alexander Schreiber

Modify GetVolumeList so output on stderr from lvs doesn't break it.

Reviewed-by: imsnah

f4bc1f2c 12/03/2007 04:03 pm Michael Hanselmann

Various code style fixes for strings.

- When line wrapping is needed, move spaces to the next line.
- Remove embedded line breaks from error messages.

Reviewed-by: schreiberal

20a0c9ef 11/14/2007 11:52 pm Guido Trotter

When an assembly error occurs log it too

Right now an assembly error produces an exception but not a log message. This
is bad because the exception suggests looking at the log, but the log itself
has a lot of errors which are not really a problem and only some which really...

7803d4d3 11/13/2007 01:11 am Iustin Pop

Fix a wrong comparison in _RecursiveAssembleBD

We want to prevent sending too many 'None' children to a device.
However, the test as it is today is wrong, as we want to test the
situation after adding a new child, and not before. This patch fixes
this by testing greater-or-equal instead of just greater....

c8a0948f 11/09/2007 04:54 pm Michael Hanselmann

Use new functions to modify /etc/hosts.

Reviewed-by: schreiberal

0834c866 11/07/2007 05:53 pm Iustin Pop

Enhance secondary node replace for drbd8

This (big) patch does two things:
- add "local disk status" to the block device checks
(BlockDevice.GetSyncStatus and the rpc calls that call this
function, and therefore cmdlib._CheckDiskConsistency)
- improve the drbd8 secondary replace operation using the above...

fc1dc9d7 11/06/2007 07:00 pm Iustin Pop

Allow DRBD8 operation without backing storage

This patch adds the following functionality:
- DRBD8 devices can assemble without local storage (done by allowing
None in the list of children, and making DRBD8 to ignore all
children if any is None)...

e739bd57 11/06/2007 07:00 pm Iustin Pop

Change the way remove children is called in bdev

For some cases, we don't have to have access to the children of a device
in order to remove them (e.g. md over lvs, or drbd over lvs). In order
to ease the removal process, skip over finding the child if it provides...

cf5a8306 11/05/2007 05:08 pm Iustin Pop

Fix a unhandled error case in device creation

The block device creation process is the following:
- device create
- device assembly (on primary or depending on dev_type, on secondary
too)
- set sync speed
- return

The problem is that device assembly after creation was not checked for...

65fe4693 11/05/2007 05:06 pm Iustin Pop

Miscellaneous style fixes

This patch fixes some minor pylint warnings (unused variables, wrong
indentation, etc.) and a real bug in the recovery for drbd8 rename
procedure.

Reviewed-by: imsnah

8fa42c7c 11/05/2007 02:43 am Guido Trotter

Make DiagnoseOS use the modified OS objects

Modify backend.py so that DiagnoseOS only returns OS objects rather than
InvalidOS errors, and make sure gnt-os understands the new objects. Also delete
the deprecated helper functions from gnt-os.

Reviewed-By: iustinp

74b4217d 11/04/2007 05:37 am Guido Trotter

Fix two typos in a doc string

Remove a wrong "i" and add a missing ")" to the DiagnoseOS function doc string.

Reviewed-By: iustinp

3f78eef2 11/02/2007 03:44 pm Iustin Pop

Implement device to instance mapping cache

Currently, troubleshooting DRBD problems involves a manual process of going
backwards from the DRBD device to the instance that owns it.

This patch adds a weak (i.e. not guaranteed to be correct or up-to-date)
cache of device to instance. The cache should be, in normal operation,...

5d0fe286 11/01/2007 07:22 pm Iustin Pop

Whitespace fixes

Reviewed-by: ultrotter

233d06c5 10/29/2007 06:02 pm Iustin Pop

Fix a non-clear error message

Reviewed-by: imsnah

a9e0c397 10/29/2007 05:49 pm Iustin Pop

Implement replace-disks for drbd8 devices

This patch adds three modes of disk replacement for drbd8:
- replace the disk on the primary node
- replace the disk on the secondary node
- replace the secondary node

It also adds some debugging code to backend.py and increments the...

f3e513ad 10/29/2007 05:44 pm Iustin Pop

Implement block device renaming

This patch add code for renaming a device; more precisely, for changing
the unique_id of the device. This means:
- logical volumes, rename the volume
- drbd8, change the remote peer

This is needed for the being able to replace disks for drbd8....

153d9724 10/25/2007 02:22 pm Iustin Pop

Modify two mirror-device related rpc calls

The two calls mirror_addchild and mirror_removechild take only one child
for addition/removal. While this is enough for our md usage, for local
disk replacement in drbd8, we need to be able to specify both the data...

fe96220b 10/19/2007 02:48 pm Iustin Pop

Abstract more strings values into constants

Currently, the disk types are defined using constants in the code.
Convert those into constants so that we can easily find them and check
their usage.

Note that we don't rename the values of the constants as they are used...

007a2f3e 10/17/2007 06:00 pm Alexander Schreiber

Patch series for reboot feature, part 1

This patch series implements the reboot command for gnt-instance. It
supports three types of reboot: soft (hypervisor reboot), hard (instance
config rebuild and reboot) and full (full instance shutdown and startup
again)....

70d9e3d8 10/16/2007 11:28 am Iustin Pop

Replace more ssh paths with proper constants

The node's ssh keys filenames are now provided as constants; this should
allow easier customization.

Also, the user's ssh key computing has been abstracted into ssh.py

Reviewed-by: imsnah

2f8598a5 10/15/2007 04:06 pm Alexander Schreiber

Trivial typing fixups.

Reviewed-by: iustinp

7900ed01 10/12/2007 05:53 pm Iustin Pop

Remove some hardcoded names/paths from backend.py

This patch does the following:
- add constants.GANETI_RUNAS = "root", which is used to compute
the homedir (and thus the .ssh directory) instead of hardcoding
"/root/.ssh" in backend.AddNode and backend.LeaveCluster...

71eca7c3 10/12/2007 03:30 pm Iustin Pop

Do not walk the whole DATA_DIR on node leave

Since we remove only files from DATA_DIR and not from subdirectories,
let's not walk the entire tree, a simple listdir suffices. Also switch
to utils.RemoveFile from simple os.unlink.

Reviewed-by: imsnah

2f31098c 10/10/2007 01:00 pm Iustin Pop

Remove the shebang from modules

Since modules are not directly executables, remove the shebang from
them. This helps with lintian warnings.

Also make the autogenerated _autoconf.py contain two comment lines at
the beginning, like the other modules.

Reviewed-by: ultrotter

3ef10550 10/08/2007 06:18 pm Michael Hanselmann

Add boot id to “gnt-node list”.

Reviewed-by: iustinp

eedbda4b 10/08/2007 11:59 am Michael Hanselmann

Add function to list files in a directory, excluding those beginning with a
dot.

Reviewed-by: iustinp

0ee60a28 10/04/2007 06:49 pm Guido Trotter

Remove redundant check.

This isdir() check leads to a broken error message. Even fixing it creates some
cases in which the error message is nebulous and unclear while removing it
makes this situation be dealt with a lot better by the _OSOndiskVersion checks....

305a7297 10/04/2007 02:51 pm Guido Trotter

Ship (and display) path for InvalidOS errors too.

- Document the expected change to errors.InvalidOS
- Always pass the additional argument
- Modify DiagnoseOS output to show the path

Reviewed-by: iustinp, imsnah

56bcd3f4 10/03/2007 05:11 pm Guido Trotter

Second part of the OS search path cleanup

Abstract the _OSSearch function, to look for an OS in the search path
Make OSFromDisk accept an optional base_dir, rather than the os_dir itself

Reviewed-by: iustinp

c26dabd7 10/03/2007 02:21 pm Guido Trotter

Make parameter os_dir mandatory for _OSOndiskversion.

First part of the OS search path cleanup. _OSOndiskversion is only ever called
once, and with that argument set, so let's make it mandatory.

Reviewed-by: iustinp

7c3d51d4 09/28/2007 12:39 pm Guido Trotter

Change constants.OS_DIR with constants.OS_SEARCH_PATH which includes all the
directories which can contain OS scripts.

The list defaults to the current one but can be changed at configure time.

Reviewed-by: imsnah

f491c3a8 09/25/2007 04:04 pm Michael Hanselmann

Rename constants for SSH init script by Iustin's request.

Reviewed-by: iustinp

c6b8baba 09/25/2007 02:41 pm Michael Hanselmann

Add configure option for init.d/ssh script. This is useful for distributions
with the script named differently than Debian.

Reviewed-by: ultrotter

9716fdce 09/17/2007 02:53 pm Iustin Pop

A few minor fixes in backend.py

This uses the recently-added Instance.FindDisk() method instead of
hard coded find-disk code.

It also renames one parameter to AddNode from ssh to sshkey in order not
to shadow the ganeti.ssh module.

Reviewed-by: imsnah

decd5f45 09/17/2007 02:52 pm Iustin Pop

Implement instance rename operation

This patch adds support for instance rename operation at all remaining
layers: RPC, OpCode/LU and CLI.

Reviewed-by: imsnah

386b57af 09/17/2007 01:52 pm Iustin Pop

Add support for rename operation in the OS API

This patch adds support for renaming at OS level. Because of this, we
need to bump up the version of the OS api from 4 to 5.

The patch also documents the new script interface in the
ganeti-os-interface(7) man page and adds a section on upgrading the OS...

72f0f7fd 09/13/2007 02:31 pm Iustin Pop

Fix the ssh change which breaks remote ssh commands

Explanation: since we use lists and not a string, every argument we give
is passed unchanged to the remote shell. So, for example, passing
'/etc/init.d/ganeti restart' to the remote shell, it will try to run the...

00003458 09/07/2007 02:30 pm Guido Trotter

Make import/export use the auxiliary ssh library to build the remote commands.

This avoids forgetting some parameters, as it's happening right now
(the correct known host file is not being passed)

In order to do so we split SSHCall into an auxiliary BuildSSHCmd which builds...

c20494cd 08/30/2007 05:32 pm Iustin Pop

Add a safety check to the backend.AddOSToInstance

This changes a ';' to '&&' to make sure we run the create script from
the correct directory.

Reviewed-by: imsnah

82122173 08/24/2007 12:20 pm Iustin Pop

Rework ssh known-hosts handling.

This changes:
- cluster setup, we no longer edit /etc/ssh/ssh_known_hosts but our
own file
- node add, we no longer remove root's known_hosts (twice)
- gnt-instance console, both the LU and the script: since now the ssh...

3ecf6786 08/14/2007 06:17 pm Iustin Pop

Style changes for pep-8 and python-3000 compliance.

This changes the raising of exceptions from:
raise Exception, value
to
raise Exception(value)

as the first form will be removed in python-3000 and the second form is
preferred now.

The changes also involve a few cases of changing from raising standard...

a0c3fea1 08/03/2007 03:35 pm Michael Hanselmann

Add instance name to LVM volume as a tag.

Reviewed-by: iustinp

e31c43f7 08/03/2007 12:18 pm Michael Hanselmann

Put synchronization speed into constant.

Reviewed-by: iustinp

098c0958 07/26/2007 02:40 pm Michael Hanselmann

Comment formatting updates.

Reviewed-by: iustinp

cb91d46e 07/25/2007 02:46 pm Iustin Pop

Since the watcher can run on all nodes, let's get rid of the cron file
handling, as it can be static and outside of ganeti.

This also means we can get rid of a lot of infrastructure too:
- the master/node config files checkers
- one rpc function

dcb93971 07/24/2007 02:45 pm Michael Hanselmann

- Implement “gnt-node volumes”
- Create all --output options using a constant
- Put node checking code from opcodes into a single function
- Do the same for output fields

Reviewed-by: iustinp

880478f8 07/23/2007 05:38 pm Iustin Pop

Big change/cleanup in relation to the master startup:
- move the master node name from the ConfigWriter to SimpleStore (all
nodes need this, and it was the only thing pulled in from the
ConfigWriter on nodes)
- fix mcpu.py and the testing w.r.t. this change; for testing, rename...

a8083063 07/16/2007 04:39 pm Iustin Pop

Initial commit.