block: add support for partial streaming
Add support for streaming data from an intermediate section of theimage chain (see patch and documentation for details).
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>...
block/vdi: Zero unused parts when allocating a new block (fix #919242)
The new block was filled with zero when it was allocated by g_malloc0,but when it was reused later and only partially used, data from thepreviously allocated block were still present and written to the new...
qcow: Return real error code in qcow_open
Signed-off-by: Li Zhi Hui <zhihuili@linux.vnet.ibm.com>Signed-off-by: Kevin Wolf <kwolf@redhat.com>
qcow: Use bdrv functions to replace file operation
Since common file operation functions lack of error detection and usemuch more I/O syscalls, so change them to bdrv series functions andreduce I/O request.
Signed-off-by: Li Zhi Hui <zhihuili@linux.vnet.ibm.com>...
block: add image streaming block job
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>Signed-off-by: Kevin Wolf <kwolf@redhat.com>
block: rate-limit streaming operations
This patch implements rate-limiting for image streaming. If we'veexceeded the bandwidth quota for a 100 ms time slice we sleep thecoroutine until the next slice begins.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>...
rbd: wire up snapshot removal and rollback functionality
Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>Reviewed-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>Signed-off-by: Kevin Wolf <kwolf@redhat.com>
block: replace unchecked strdup/malloc/calloc with glib
Most of the codebase as been converted to use glib memory allocationfunctions. There are still a few instances of malloc/calloc in theblock layer and qemu-io. Replace them, especially since they do not...
prepare for future GPLv2+ relicensing
All files under GPLv2 will get GPLv2+ changes starting tomorrow.event_notifier.c and exec-obsolete.h were only ever touched by Red Hatemployees and can be relicensed now.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>...
vvfat: avoid leaking file descriptor in commit_one_file()
Reported-by: Dr David Alan Gilbert <davidagilbert@uk.ibm.com>Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
move corking functions to osdep.c
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
nbd: switch to asynchronous operation
nbd: split requests
qemu-nbd has a limit of slightly less than 1M per request. Workaround this in the nbd block driver.
nbd: allow multiple in-flight requests
Allow sending up to 16 requests, and drive the replies to the coroutinethat did the request. The code is written to be exactly the same asbefore this patch when MAX_NBD_REQUESTS == 1 (modulo the extra mutexand state)....
nbd: add support for NBD_CMD_FLAG_FUA
nbd: add support for NBD_CMD_FLUSH
nbd: add support for NBD_CMD_TRIM
sheepdog: move coroutine send/recv function to generic code
Outside coroutines, avoid busy waiting on EAGAIN by temporarilymaking the socket blocking.
The API of qemu_recvv/qemu_sendv is slightly different fromdo_readv/do_writev because they do not handle coroutines. It...
qcow2: Allow >4 GB VM state
This is a compatible extension to the snapshot header format that allowssaving a 64 bit VM state size.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
block/cow: Return real error code
block: qemu_aio_get does not return NULL
Initially done with the following semantic patch:
rule1 expression E;statement S;@@ E = qemu_aio_get (...);(- if (E == NULL) { ... } |- if (E) { <... S ...> })
rule1
which however missed occurrences in linux-aio.c and posix-aio-compat.c....
block/qcow2.c: call qcow2_free_snapshots in the function of qcow2_close
Signed-off-by: Li Zhi Hui <zhihuili@linux.vnet.ibm.com>Reviewed-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>Signed-off-by: Kevin Wolf <kwolf@redhat.com>
rbd: always set out parameter in qemu_rbd_snap_list
The caller expects psn_tab to be NULL when there are no snapshots oran error occurs. This results in calling g_free on an invalid address.
Reported-by: Oliver Francke <Oliver@filoo.de>Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>...
block: bdrv_aio_* do not return NULL
rule1 expression E;statement S;@@ E =( bdrv_aio_readv | bdrv_aio_writev | bdrv_aio_flush | bdrv_aio_discard | bdrv_aio_ioctl) (...);(- if (E == NULL) { ... }...
fix typo: delete redundant semicolon
Double semicolons should be single.
Signed-off-by: Dong Xu Wang <wdongxu@linux.vnet.ibm.com>Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Merge remote-tracking branch 'kwolf/for-anthony' into staging
coroutine: add qemu_co_queue_restart_all()
It's common to wake up all waiting coroutines. Introduce theqemu_co_queue_restart_all() function to do this instead of looping overqemu_co_queue_next() in every caller.
cow: use bdrv_co_is_allocated()
Now that bdrv_co_is_allocated() is available we can use it instead ofthe synchronous bdrv_is_allocated() interface. This is a follow-up thatKevin Wolf <kwolf@redhat.com> pointed out after applying the series thatintroduces bdrv_co_is_allocated()....
qed: convert to .bdrv_co_is_allocated()
The bdrv_qed_is_allocated() function is a synchronous wrapper aroundqed_find_cluster(), which performs the cluster lookup. In order toconvert the synchronous function to a coroutine function we yieldinstead of using qemu_aio_wait(). Note that QED's cache is already safe...
block: convert qcow2, qcow2, and vmdk to .bdrv_co_is_allocated()
The qcow2, qcow, and vmdk block drivers are based on coroutines. They have acoroutine mutex which protects internal state. We can convert the.bdrv_is_allocated() function to .bdrv_co_is_allocated() by holding the mutex...
vvfat: convert to .bdrv_co_is_allocated()
It is trivial to switch from the synchronous .bdrv_is_allocated()interface to .bdrv_co_is_allocated() since vvfat_is_allocated() does notblock.
vdi: convert to .bdrv_co_is_allocated()
It is trivial to switch from the synchronous .bdrv_is_allocated()interface to .bdrv_co_is_allocated() since vdi_is_allocated() does notblock.
cow: convert to .bdrv_co_is_allocated()
The cow block driver does not keep internal state for cluster lookups.This means it is safe to perform cluster lookups in coroutine contextwithout risk of race conditions that corrupt internal state.
qcow2: Update snapshot table information at once
Failing in the middle wouldn't help with the integrity of the image, sodoing everything in a single request seems better.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>Reviewed-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
qcow2: Cleanups and memleak fix in qcow2_snapshot_create
sn->id_str could be leaked before this. The rest of this patch changescomments, fixes coding style or removes checks that are unnecessary withg_malloc.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>...
qcow2: Rework qcow2_snapshot_create error handling
Increase refcounts only after allocating a new L1 table has succeeded inorder to make leaks less likely. If writing the snapshot table fails,revert in-memory state to be consistent with that on disk.
While at it, make it return the real error codes instead of -1....
qcow2: Return real error in qcow2_snapshot_goto
Besides fixing the return code, this adds some comments that make clearhow the code works and that it potentially breaks images if we fail inthe wrong place. Actually fixing this is left for the next patch....
qcow2: Fix order of refcount updates in qcow2_snapshot_goto
The refcount updates must be moved so that in the worst case we can getcluster leaks, but refcounts may never be too low.
qcow2: Fix order in qcow2_snapshot_delete
First the snapshot must be deleted and only then the refcounts can bedecreased.
qcow2: Fix error path in qcow2_snapshot_load_tmp
If the bdrv_read() of the snapshot's L1 table fails, return the righterror code and make sure that the old L1 table is still loaded and wedon't break the BlockDriverState completely.
block: Use bdrv functions to replace file operation in cow.c
Since common file operation functions lack of error detection,so change them to bdrv series functions.
Signed-off-by: Li Zhi Hui <zhihuili@linux.vnet.ibm.com>Reviewed-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>...
block: Add coroutine_fn marker to coroutine functions
Looks better when reviewing these source files.
Signed-off-by: Dong Xu Wang <wdongxu@linux.vnet.ibm.com>Reviewed-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>Signed-off-by: Kevin Wolf <kwolf@redhat.com>
qcow2: Return real error code in qcow2_read_snapshots
qcow2: Return real error code in qcow2_write_snapshots
Doesn't immediately fix anything as the callers don't use the returnvalue, but they will be fixed next.
qed: adjust the way to get nb_sectors
This patch is only to refactor some lines of codes to get better and more robust codes.
As you have seen, in qed_read_table_cb() it's nice touse qiov->size because that function doesn't obviously use a singlestruct iovec....
qcow2: avoid reentrant bdrv_read() in copy_sectors()
A BlockDriverState should not issue requests on itself through thepublic block layer interface. Nested, or reentrant, requests areproblematic because they do I/O throttling and request tracking twice....
qcow2: Unlock during COW
Unlocking during COW allows for more parallelism. One change it requires isthat buffers are dynamically allocated instead of just using a per-imagebuffer.
While touching the code, drop the synchronous qcow2_read() function and replace...
fix spelling in block sub directory
Cc: Kevin Wolf <kwolf@redhat.com>Signed-off-by: Dong Xu Wang <wdongxu@linux.vnet.ibm.com>Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
vpc: Add missing error handling in alloc_block
vvfat: Add migration blocker
vvfat caches more or less everything when in writable mode. For migrationto work, it would have to be invalidated. Block migration for now whenin writable mode (default is readonly).
vdi: Fix memory leak
The block map is allocated in vdi_open, but was never freed.
vpc: Add migration blocker
vpc caches the BAT. For migration to work, it would have to beinvalidated. Block migration for now.
vmdk: Add migration blocker
VMDK caches L2 tables. For migration to work, they would have to beinvalidated. Block migration for now.
vdi: Add migration blocker
vdi caches the block map. For migration to work, it would have to beinvalidated. Block migration for now.
qcow: Add migration blocker
qcow caches L2 tables. For migration to work, they would have to beinvalidated. Block migration for now.
sheepdog: Avoid deadlock in error path
s->lock should be unlocked before leaving add_aio_request.
Signed-off-by: Dong Xu Wang <wdongxu@linux.vnet.ibm.com>Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Include zlib.h using #include <>
zlib.h is not a local include file, therefore it should be includedusing <> instead of "".
Signed-off-by: Stefan Weil <sw@weilnetz.de>Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
qcow2: implement bdrv_invalidate_cache (v2)
We don't reopen the actual file, but instead invoke the close and open routines.We specifically ignore the backing file since it's contents are read-only andtherefore immutable.
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
qed: add migration blocker (v2)
Now when you try to migrate with qed, you get:
(qemu) migrate tcp:localhost:1025Block format 'qed' used by device 'ide0-hd0' does not support feature 'live migration'(qemu)
block: Rename bdrv_co_flush to bdrv_co_flush_to_disk
There are two different types of flush that you can do: Flushing one level upto the OS (i.e. writing data to the host page cache) or flushing it all the waydown to the disk. The existing functions flush to the disk, reflect this in the...
block: Introduce bdrv_co_flush_to_os
qcow2 has a writeback metadata cache, so flushing a qcow2 image actuallyconsists of writing back that cache to the protocol and only then flushes theprotocol in order to get everything stable on disk.
This introduces a separate bdrv_co_flush_to_os to reflect the split....
block: Fix vpc initialization of the Dynamic Disk Header
The Data Offset field in the Dynamic Disk Header is an 8 byte field.Although the specification (2006-10-11) gives an example of initializingonly the first 4 bytes, images generated by Microsoft on Windows initialize...
vvfat: Fix read-write mode
vvfat used to directly call into the qcow2 block driver instead of using theblock.c wrappers. With the coroutine conversion, this stopped working.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
vvfat: reorganize computation of disk geometry
First determine FAT12/16/32, then compute geometry from that for bothFDD and HDD. For 1.44MB floppies, and 2.88MB floppies using FAT16,change to 1 sector/cluster. The default remains 2.88MB with FAT12and 2 sectors/cluster. Both DOS and mkdosfs by default format a 2.88MB...
vvfat: do not hardcode sector counts in error message
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>Signed-off-by: Kevin Wolf <kwolf@redhat.com>
vvfat: unify and correct computation of sector count
The sector count is stored in the partition and hence must not include thesectors before its start. At the same time, remove the useless specialcasing for 1.44 MB floppies. This fixes fsck on VVFAT hard disks,...
vvfat: need to use first_sectors_number to distinguish fdd/hdd
This is consistent with what "real" floppies have, so file(1)now actually recognizes the VVFAT image as a 1.44 MB floppy.
vvfat: do not fail if the disk has spare sectors
If the number of "faked sectors" + the number of sectors that arepart of a cluster does not sum up to the total number of sectors,qemu-img convert fails. Read these spare sectors as all zeros.
vvfat: fix out of bounds array_get usage
When reading the address of the first free entry, you cannotuse array_get without first marking all entries as occupied.
This is visible if you change the sectors per cluster on afloppy from 2 to 1.
block/cloop: Use g_free instead of free
Fix mismatching allocation and deallocation: g_free should be used to pair withg_malloc.
Reviewed-by: Andreas Färber <afaerber@suse.de>Reviewed_by: Ray Wang <raywang@linux.vnet.ibm.com>Signed-off-by: Dong Xu Wang <wdongxu@linux.vnet.ibm.com>...
block/cloop: Fix coding style
Fix coding style in block/cloop.c.
Reviewed-by: Andreas Färber <afaerber@suse.de>Reviewed_by: Ray Wang <raywang@linux.vnet.ibm.com>Signed-off-by: Dong Xu Wang <wdongxu@linux.vnet.ibm.com>Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Conflicts: block/vmdk.c
Merge remote-tracking branch 'stefanha/trivial-patches' into staging
Teach block/vdi about "discarded" (no longer allocated) blocks
An entry in the VDI block map will hold an offset to the actual block ifthe block is allocated, or one of two specially-interpreted values ifnot allocated. Using VirtualBox terminology, value VDI_IMAGE_BLOCK_FREE...
qcow: Fix bdrv_write_compressed error handling
vmdk: Fix use of uninitialised value
In error cases, cid is never set.
vmdk: Improve error handling
Return the right error values in some more places.
vmdk: Fix possible segfaults
Data we read from the disk isn't necessarily null terminated and may notcontain the string we're looking for. The code needs to be a bit more carefulhere.
block: fix qcow2_co_flush deadlock
If qcow2_cache_flush failed, s->lock will not be unlock.
iSCSI block driver
This provides built-in support for iSCSI to QEMU.
This has the advantage that the iSCSI devices need not be made visible to the host, which is useful if you have very many virtual machines and very many iscsi devices.It also has the benefit that non-root users of QEMU can access iSCSI devices across the network without requiring root privilege on the host....
vmdk: remove unneeded variable assignment
Spotted by Clang Analyzer
Signed-off-by: Pavel Borzenkov <pavel.borzenkov@gmail.com>Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
vmdk: vmdk_read_cid returns garbage if p_name is NULL
qed: don't pass NULL to memcpy
[Note this memcpy call has always been safe because the length will be 0when the pointer is NULL]
qed: remove unneeded variable assignment
'ret' is unconditionally overwitten by qed_read_l1_table_sync()
block: take lock around bdrv_read implementations
This does the first part of the conversion to coroutines, bywrapping bdrv_read implementations to take the mutex.
Drivers that implement bdrv_read rather than bdrv_co_readv canthen benefit from asynchronous operation (at least if the underlying...
block: take lock around bdrv_write implementations
This does the first part of the conversion to coroutines, bywrapping bdrv_write implementations to take the mutex.
Drivers that implement bdrv_write rather than bdrv_co_writev canthen benefit from asynchronous operation (at least if the underlying...
block: change flush to co_flush
Since coroutine operation is now mandatory, convert all bdrv_flushimplementations to coroutines. For qcow2, this means taking the lock.Other implementations are simpler and just forward bdrv_flush to theunderlying protocol, so they can avoid the lock....
block: change discard to co_discard
Since coroutine operation is now mandatory, convert both bdrv_discardimplementations to coroutines. For qcow2, this means taking the lockaround the operation. raw-posix remains synchronous.
The bdrv_discard callback is then unused and can be eliminated....
block: drop redundant bdrv_flush implementation
Block drivers now only need to provide either of .bdrv_co_flush,.bdrv_aio_flush() or for legacy drivers .bdrv_flush(). Removethe redundant .bdrv_flush() implementations.
[Paolo Bonzini: change raw driver to bdrv_co_flush]...
block: add bdrv_co_discard and bdrv_aio_discard support
This similarly adds support for coroutine and asynchronous discard.
qcow2: Fix bdrv_write_compressed error handling
If during allocation of compressed clusters the cluster was already allocateduncompressed, fail and properly release the l2_table (the latter avoids afailed assertion).
While at it, make it return some real error numbers instead of -1....
vmdk: fix return values of vmdk_parent_open
While vmdk_open_desc_file (touched by the patch) correctly changed -1to -EINVAL, vmdk_open did not. Fix it directly in vmdk_parent_open.
vmdk: clean up open
Move vmdk_parent_open to vmdk_open. There's another path howvmdk_parent_open can be reached:
vmdk_parse_extents() -> vmdk_open_sparse() -> vmdk_open_vmdk4() -> vmdk_open_desc_file().
If that can happen, however, the code is bogus. vmdk_parent_open...
block: add a CoMutex to synchronous read drivers
The big conversion of bdrv_read/write to coroutines caused the twohomonymous callbacks in BlockDriver to become reentrant. It goeslike this:
1) bdrv_read is now called in a coroutine, and calls bdrv_read or...
sheepdog: add coroutine_fn markers
This makes the following patch easier to review.
Cc: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>Signed-off-by: Kevin Wolf <kwolf@redhat.com>
raw-posix: remove bdrv_read()/bdrv_write()
Block drivers only need to provide one of sync, aio, or coroutineinterfaces. Since raw-posix.c provides aio interfaces, simply drop thesynchronous interfaces since they can be emulated using aio andcoroutines....
block: use coroutine interface for raw format
The raw format delegates all operations to bs->file (the protocol).Previously this block driver exposed both sync and aio interfaces.Since the block layer now works in terms of coroutines, expose thecoroutine interfaces and drop the others. This avoids unnecessary...
block/qcow: Fix use of free() instead of g_free()
cppcheck reported this error:
qemu/block/qcow.c:599: error: Mismatching allocation and deallocation: cluster_data
Signed-off-by: Stefan Weil <sw@weilnetz.de>Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
sheepdog: correct spelling
Reviewed-by: Andreas Färber <afaerber@suse.de>Signed-off-by: Dong Xu Wang <wdongxu@linux.vnet.ibm.com>Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>