Commit Graph

64139 Commits

Author SHA1 Message Date
Peter Maydell d21ee59ae5 nbd patches for 2018-10-03
Fix bugs in NBD_CMD_CACHE, drop support for oldstyle NBD server,
 minor build and doc fixes
 
 - Denis V. Lunev: nbd: fix NBD_CMD_CACHE negitiation... [retitled]
 - Vladimir Sementsov-Ogievskiy: 0/2 server: drop old-style negotiation
 - Eric Blake: qemu-nbd: Document --tls-creds
 - Vladimir Sementsov-Ogievskiy: nbd/server: fix NBD_CMD_CACHE
 - Peter Maydell: nbd: Don't take address of fields in packed structs
 -----BEGIN PGP SIGNATURE-----
 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
 
 iQEcBAABCAAGBQJbtiF0AAoJEKeha0olJ0NqqgsH/RKL+SPl8S4xslZ8pTcXL2OW
 AgMZInxKAe3MDX6NiCYFDWU4/32NNzPluWMh7DGu3mqFQiVySz9oAz2csZ4Fd5nm
 NfY6Ewu64HF/REN7dNLfyrbmfzD3PdCUAa2PsasH1wzKArTklyfJH/QeOHbJ60H8
 5AjthM/ksFNSAZXp+R+gzr2zGHN1cXwNBn87tKndPYaiIc95azPSZOnQpLLN9gfH
 YwxCKk+opeEJNMuXf/PdkksbxqNxHsBCYSAL9NzjzS/+T8GIpHQAZvKGSk6FKDp+
 x+KcC2pL7Lmu6iJWGIo5hGSSsBP1aXp0MJ/dUD+jo13LWNwzGFmovNI4pUuEK2A=
 =0s7A
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2018-10-03-v2' into staging

nbd patches for 2018-10-03

Fix bugs in NBD_CMD_CACHE, drop support for oldstyle NBD server,
minor build and doc fixes

- Denis V. Lunev: nbd: fix NBD_CMD_CACHE negitiation... [retitled]
- Vladimir Sementsov-Ogievskiy: 0/2 server: drop old-style negotiation
- Eric Blake: qemu-nbd: Document --tls-creds
- Vladimir Sementsov-Ogievskiy: nbd/server: fix NBD_CMD_CACHE
- Peter Maydell: nbd: Don't take address of fields in packed structs

# gpg: Signature made Thu 04 Oct 2018 15:19:32 BST
# gpg:                using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>"
# gpg:                 aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>"
# gpg:                 aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2018-10-03-v2:
  nbd: fix NBD_FLAG_SEND_CACHE value
  nbd/server: drop old-style negotiation
  qemu-nbd: drop old-style negotiation
  qemu-nbd: Document --tls-creds
  nbd/server: fix NBD_CMD_CACHE
  nbd: Don't take address of fields in packed structs

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-10-05 14:52:54 +01:00
Peter Maydell 638ad4ad17 Python queue, 2018-10-03
* Remove fixed serial device errors from device-crash-test
 * Remove unnecessary Python 2.6 compatibility code
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABCAAGBQJbtXg+AAoJECgHk2+YTcWm65wP/3LfgSzM1znaWa3RgNNN3rKB
 FA+C6lT6LM+0TOf/uwoM2Q0vfgo3oEE1WbrBCHjI0LnsV0D94KOOkd6oPuFGdhJg
 HqPDAhJHa/FDx+et4QV81uKVKhyl/9xQ3j5elgydaGRMeWDZmhYM6nGL7M3+LWwh
 ymNDKbfMnfESvPvIYW/VS8LDtU1V88yf+dGtK3lyyMgDIp2OiaesGwYAoa695Djg
 jbZq3meU7vRNo6C92FC1KP363x2KLzpuiEmpIvPMK37ScpS8O4QGzNmQGP2Z9UM1
 Ow+/QJIpNrkqU6B3SObxPL7BhdQZw/GLiSSt0/svaXDKqvcU87uyM14hl1SB9mbU
 SMQXKumTOwO/UQpfl/MNPQPHxBDSoMNSI7Rbzf7QjhnqYp6+1o5PffMbzPa/4ThJ
 SMdIfBJqe4Bc/pl0j2O0Cao3nhpvmBBuq5cJk+4z7zKAr2jenXr4EY0Uwl8MYTY+
 S5vzgjKjVtSQP0XhSFtsf+5M3ztfUhXsyX95Xi4qh1waAHsJTzMHIu2urPOnvDzO
 RodfHcO5Vcnw7C9iSquqNZkiL9cbkMLpIojwtp9PGeiUIOyZSm9fVEzt0ZVIyDIW
 jxChIAgn3lHSovVaWdL9tnmYZyOnLyKThPfQo1inFcFR2AAhvgqR9yGeKGIZ2KEr
 pYm3E0dnJzR8o0lC7YCX
 =h6yh
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ehabkost/tags/python-next-pull-request' into staging

Python queue, 2018-10-03

* Remove fixed serial device errors from device-crash-test
* Remove unnecessary Python 2.6 compatibility code

# gpg: Signature made Thu 04 Oct 2018 03:17:34 BST
# gpg:                using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>"
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF  D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/python-next-pull-request:
  scripts/device-crash-test: Remove entries for serial devices
  Revert "tests: migration/guestperf Python 2.6 argparse compatibility"
  Revert "docker.py: Python 2.6 argparse compatibility"
  device-crash-test: No need for sys.path hack

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-10-05 12:45:10 +01:00
Denis V. Lunev df91328ada nbd: fix NBD_FLAG_SEND_CACHE value
Commit bc37b06a5 added NBD_CMD_CACHE support, but used the wrong value
for NBD_FLAG_SEND_CACHE flag for negotiation. That commit picked bit 8,
which had already been assigned by the NBD specification to mean
NBD_FLAG_CAN_MULTI_CONN, and which was already implemented in the
Linux kernel as a part of stable userspace-kernel API since 4.10:

"bit 8, NBD_FLAG_CAN_MULTI_CONN: Indicates that the server operates
entirely without cache, or that the cache it uses is shared among all
connections to the given device. In particular, if this flag is
present, then the effects of NBD_CMD_FLUSH and NBD_CMD_FLAG_FUA
MUST be visible across all connections when the server sends its reply
to that command to the client. In the absense of this flag, clients
SHOULD NOT multiplex their commands over more than one connection to
the export.
...
bit 10, NBD_FLAG_SEND_CACHE: documents that the server understands
NBD_CMD_CACHE; however, note that server implementations exist
which support the command without advertising this bit, and
conversely that this bit does not guarantee that the command will
succeed or have an impact."

Consequences:
- a client trying to use NBD_CMD_CACHE per the NBD spec will not
see the feature as available from a qemu 3.0 server (not fatal,
clients already have to be prepared for caching to not exist)
- a client accidentally coded to the qemu 3.0 bit value instead
of following the spec may interpret NBD_CMD_CACHE as being available
when it is not (probably not fatal, the spec says the server should
gracefully fail unknown commands, and that clients of NBD_CMD_CACHE
should be prepared for failure even when the feature is advertised);
such clients are unlikely (perhaps only in unreleased Virtuozzo code),
and will disappear over time
- a client prepared to use multiple connections based on
NBD_FLAG_CAN_MULTI_CONN may cause data corruption when it assumes
that caching is consistent when in reality qemu 3.0 did not have
a consistent cache. Partially mitigated by using read-only
connections (where nothing needs to be flushed, so caching is
indeed consistent) or when using qemu-nbd with the default -e 1
(at most one client at a time); visible only when using -e 2 or
more for a writable export.

Thus the commit fixes negotiation flag in QEMU according to the
specification.

Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
CC: Valery Vdovin <valery.vdovin@acronis.com>
CC: Eric Blake <eblake@redhat.com>
CC: Paolo Bonzini <pbonzini@redhat.com>
CC: qemu-stable@nongnu.org
Message-Id: <20181004100313.4253-1-den@openvz.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: enhance commit message, add defines for unimplemented flags]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-10-04 09:08:56 -05:00
Thomas Huth cd0c3da730 scripts/device-crash-test: Remove entries for serial devices
The problem with the various serial devices has been fixed a while
ago in commit 47c4f85a0c ("hw/char/serial:
Allow disconnected chardevs") already, so we can remove these entries
from the "ignore" list in the device-crash-test script now.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <1538403190-27146-1-git-send-email-thuth@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-10-03 23:08:51 -03:00
Eduardo Habkost afb6249600 Revert "tests: migration/guestperf Python 2.6 argparse compatibility"
This reverts commit 0ea47d0f36.

scripts/argparse.py was removed from the tree, so we don't
need this hack anymore.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180618225131.13113-4-ehabkost@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-10-03 23:08:51 -03:00
Eduardo Habkost ae68fdaba9 Revert "docker.py: Python 2.6 argparse compatibility"
This reverts commit c2d3189667.

scripts/argparse.py was removed from the tree, so we don't need
this hack anymore.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180618225131.13113-3-ehabkost@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-10-03 23:08:51 -03:00
Eduardo Habkost 8d0ea57ad4 device-crash-test: No need for sys.path hack
The device-crash-test script is already inside the 'scripts'
directory, there's no need to add the directory manually to
sys.path.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180618225131.13113-2-ehabkost@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-10-03 23:08:51 -03:00
Vladimir Sementsov-Ogievskiy 7f7dfe2a53 nbd/server: drop old-style negotiation
After the previous commit, nbd_client_new's first parameter is always
NULL. Let's drop it with all corresponding old-style negotiation code
path which is unreachable now.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20181003170228.95973-3-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: re-wrap short line]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-10-03 15:52:32 -05:00
Vladimir Sementsov-Ogievskiy f5cd0bb517 qemu-nbd: drop old-style negotiation
Use new-style negotiation always, with default "" (empty) export name
if it is not specified with '-x' option.

qemu as client can manage either style since 2.6.0, commit 69b49502d8

For comparison:

nbd 3.10 dropped oldstyle long ago (Mar 2015):
https://github.com/NetworkBlockDevice/nbd/commit/36940193

nbdkit 1.3 switched its default to newstyle (Jan 2018):
https://github.com/libguestfs/nbdkit/commit/b2a8aecc
https://github.com/libguestfs/nbdkit/commit/8158e773

Furthermore, if a client that only speaks oldstyle still needs to
communicate to qemu, nbdkit remains available to perform the
translation between the two protocols.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20181003170228.95973-2-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: enhance commit message]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-10-03 15:48:31 -05:00
Eric Blake f7812df77d qemu-nbd: Document --tls-creds
Commit 145614a1 introduced --tls-creds and documented it in
qemu-nbd.texi, but forgot to document it in 'qemu-nbd --help'.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20181003180426.602765-1-eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2018-10-03 15:46:52 -05:00
Vladimir Sementsov-Ogievskiy 2f454defc2 nbd/server: fix NBD_CMD_CACHE
We should not go to structured-read branch on CACHE command, fix that.

Bug introduced in bc37b06a5c "nbd/server: introduce NBD_CMD_CACHE"
with the whole feature and affects 3.0.0 release.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
CC: qemu-stable@nongnu.org
Message-Id: <20181003144738.70670-1-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: commit message typo fix]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-10-03 09:58:43 -05:00
Peter Maydell 80c7c2b00d nbd: Don't take address of fields in packed structs
Taking the address of a field in a packed struct is a bad idea, because
it might not be actually aligned enough for that pointer type (and
thus cause a crash on dereference on some host architectures). Newer
versions of clang warn about this. Avoid the bug by not using the
"modify in place" byte swapping functions.

This patch was produced with the following spatch script:
@@
expression E;
@@
-be16_to_cpus(&E);
+E = be16_to_cpu(E);
@@
expression E;
@@
-be32_to_cpus(&E);
+E = be32_to_cpu(E);
@@
expression E;
@@
-be64_to_cpus(&E);
+E = be64_to_cpu(E);
@@
expression E;
@@
-cpu_to_be16s(&E);
+E = cpu_to_be16(E);
@@
expression E;
@@
-cpu_to_be32s(&E);
+E = cpu_to_be32(E);
@@
expression E;
@@
-cpu_to_be64s(&E);
+E = cpu_to_be64(E);

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20180927164200.15097-1-peter.maydell@linaro.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: rebase, and squash in missed changes]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-10-03 09:58:43 -05:00
Peter Maydell b8e5671a8c chardev patches
-----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJbtKCeAAoJENro4Ql1lpzlpSAQALQLmMMuzYwUkpC7J5rxvlYv
 bKpdi3urdXxQ2zG+QDErCQEsJdGefmJkcT236qRyWR2Ol5J0x5aOA/vZO3EDRGbH
 uEvuaJt0WamPOT8H2JKGfYyp6dT+VvO7ZIJqeCWGFyMy1LJaJR6eYnEjr2Z0eB/J
 s8MhIPTw5xl4x44Bi9/52nkk3HZlpZF36CvdQA+rGVvz1E/MFbDECgV6xMGMmTPw
 P95pgK73JWB+B+6IZcEDUrL42xH3SBqm/ty9r8kT45NGui1l6CdwjEck4MDnafdc
 bsGTMCJn35B0rPx/vPd9M/ocw4q1Ql2ErvOiMo6Kx6d98Y7O7+jf1NF3uZfT19fc
 Y1/ysv2a7BMHF81iJcR4xYQGs5s1xenxm2pK1A1dxz5Ltwbw/3ns+sDCkQvZww9q
 UAc9y+fywmqJVFhX/pF7DLkpeKa2LPviBiJkEKErrrgXv1bU5TcR18NkDcXl4eR6
 Mi9mjyx868lotlHOwcbcdZGvrY82HFToxTuU8tleNKsdAE9q8zT9Iocg6Q5xAvH2
 2gwW4isY0RvGvhWwabAhOVw1kIsPD2qICzMUZGKtYA6o4whbY4uvxP+EVtKuy8KP
 OIyMd4YcGvS065QvHzwCu8U9U7eJPZP9cbrn+iP93gBGBHiL+w9hhU10sEiTUYw+
 JqwsmDWVhUx9RIdftiZn
 =xdVN
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/elmarco/tags/chardev-pull-request' into staging

chardev patches

# gpg: Signature made Wed 03 Oct 2018 11:57:34 BST
# gpg:                using RSA key DAE8E10975969CE5
# gpg: Good signature from "Marc-André Lureau <marcandre.lureau@redhat.com>"
# gpg:                 aka "Marc-André Lureau <marcandre.lureau@gmail.com>"
# Primary key fingerprint: 87A9 BD93 3F87 C606 D276  F62D DAE8 E109 7596 9CE5

* remotes/elmarco/tags/chardev-pull-request:
  chardev: use a child source for qio input source
  chardev: mark the calls that allow an implicit mux monitor
  char.h: fix gtk-doc comment style
  chardev: unref if underlying chardev has no parent
  chardev: remove qemu_chr_fe_read_all() counter
  chardev: avoid crash if no associated address

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-10-03 14:07:49 +01:00
Marc-André Lureau a7077b8e35 chardev: use a child source for qio input source
GLib child source were added with version 2.28. We can use them now
that we bumped our requirement to 2.40.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-10-03 14:45:05 +04:00
Marc-André Lureau 95e30b2a13 chardev: mark the calls that allow an implicit mux monitor
This is mostly for readability of the code. Let's make it clear which
callers can create an implicit monitor when the chardev is muxed.

This will also enforce a safer behaviour, as we don't really support
creating monitor anywhere/anytime at the moment. Add an assert() to
make sure the programmer explicitely wanted that behaviour.

There are documented cases, such as: -serial/-parallel/-virtioconsole
and to less extent -debugcon.

Less obvious and questionable ones are -gdb, SLIRP -guestfwd and Xen
console. Add a FIXME note for those, but keep the support for now.

Other qemu_chr_new() callers either have a fixed parameter/filename
string or do not need it, such as -qtest:

* qtest.c: qtest_init()
  Afaik, only used by tests/libqtest.c, without mux. I don't think we
  support it outside of qemu testing: drop support for implicit mux
  monitor (qemu_chr_new() call: no implicit mux now).

* hw/
  All with literal @filename argument that doesn't enable mux monitor.

* tests/
  All with @filename argument that doesn't enable mux monitor.

On a related note, the list of monitor creation places:

- the chardev creators listed above: all from command line (except
  perhaps Xen console?)

- -gdb & hmp gdbserver will create a "GDB monitor command" chardev
  that is wired to an HMP monitor.

- -mon command line option

From this short study, I would like to think that a monitor may only
be created in the main thread today, though I remain skeptical :)

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
2018-10-03 14:45:05 +04:00
Marc-André Lureau 5662576ad0 char.h: fix gtk-doc comment style
Fix up conformance to GTK-Doc function comment style, as documented in
https://developer.gnome.org/gtk-doc-manual/stable/documenting_symbols.html.en

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
2018-10-03 14:45:05 +04:00
Marc-André Lureau 425d8a4e70 chardev: unref if underlying chardev has no parent
It's possible to write code creating a chardev backend that is not
registered. When it is not user-created, it makes sense to keep it
hidden. Let the associated frontend destroy it also in this case.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-10-03 14:45:04 +04:00
Marc-André Lureau 43182856c5 chardev: remove qemu_chr_fe_read_all() counter
There is no obvious reason to have a loop counter. This limits from
reading several megabytes large buffers in one go, since socket
read/write usually have a limit.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-03 14:45:04 +04:00
Marc-André Lureau 90a6d17bed chardev: avoid crash if no associated address
A socket chardev may not have associated address (when adding client
fd manually for example). But on disconnect, updating socket filename
expects an address and may lead to this crash:

  Thread 1 "qemu-system-x86" received signal SIGSEGV, Segmentation fault.
  0x0000555555d8c70c in SocketAddress_to_str (prefix=0x555556043062 "disconnected:", addr=0x0, is_listen=false, is_telnet=false) at /home/elmarco/src/qq/chardev/char-socket.c:388
  388	    switch (addr->type) {
  (gdb) bt
  #0  0x0000555555d8c70c in SocketAddress_to_str (prefix=0x555556043062 "disconnected:", addr=0x0, is_listen=false, is_telnet=false) at /home/elmarco/src/qq/chardev/char-socket.c:388
  #1  0x0000555555d8c8aa in update_disconnected_filename (s=0x555556b1ed00) at /home/elmarco/src/qq/chardev/char-socket.c:419
  #2  0x0000555555d8c959 in tcp_chr_disconnect (chr=0x555556b1ed00) at /home/elmarco/src/qq/chardev/char-socket.c:438
  #3  0x0000555555d8cba1 in tcp_chr_hup (channel=0x555556b75690, cond=G_IO_HUP, opaque=0x555556b1ed00) at /home/elmarco/src/qq/chardev/char-socket.c:482
  #4  0x0000555555da596e in qio_channel_fd_source_dispatch (source=0x555556bb68b0, callback=0x555555d8cb58 <tcp_chr_hup>, user_data=0x555556b1ed00) at /home/elmarco/src/qq/io/channel-watch.c:84

Replace filename with a generic "disconnected:socket" in this case.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-10-03 14:45:04 +04:00
Peter Maydell dafd950536 * configure fix for environment variables (Daniel)
* fix memory leaks (Alex)
 * x86_64 MTTCG fixes (Emilio)
 * introduce atomic64 (Emilio)
 * Fix for virtio hang (Fam, myself)
 * SH serial port fix (Geert)
 * Deprecate rotation_rate for scsi-block (Fam)
 * Extend memory-backend-file availability to all POSIX hosts (Hikaru)
 * Memory API cleanups and fixes (Igor, Li Qiang, Peter, Philippe)
 * MSI/IOMMU fix (Jan)
 * Socket reconnection fixes (Marc-André)
 * icount fixes (Emilio, myself)
 * QSP fixes for Coverity (myself)
 * Some record/replay improovements (Pavel)
 * Packed struct fixes (Peter)
 * Windows dump fixes and elf2dmp (Viktor)
 * kbmclock fix (Yongji)
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQEcBAABAgAGBQJbs6coAAoJEL/70l94x66DaL0IAISiRZcm7SMFTUafivyzQ9Ao
 vk2SZ64/BUmDI5q5t30NGiVkMzAc0qDunRSqD4FnIvhGl8phFSSYqaN28JFLe4l1
 JhX7FdLQgeevYY35hEPjpCEOAR7WD116/NaZ/UZ+7zZ4Z+CtcCEXZefb4dD9vijj
 M/rH7vXJsulSb7q2Np3hhbai/GL7ZvNURaHOXZpuPE2aJGAcSXhYtAbGHPJ4NKgn
 qjP3AGTose8cRD0u5smY0JnyL5vcF606+dupIUsnciDC3wF1SPusTMwLRLoEGNA4
 lmxTdGbxvvM1TDu/mY70WYXJ2ujC9Suhj1jkftgDTWRZqwQ9N/B4eB2JcQ9WbhQ=
 =SzeA
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* configure fix for environment variables (Daniel)
* fix memory leaks (Alex)
* x86_64 MTTCG fixes (Emilio)
* introduce atomic64 (Emilio)
* Fix for virtio hang (Fam, myself)
* SH serial port fix (Geert)
* Deprecate rotation_rate for scsi-block (Fam)
* Extend memory-backend-file availability to all POSIX hosts (Hikaru)
* Memory API cleanups and fixes (Igor, Li Qiang, Peter, Philippe)
* MSI/IOMMU fix (Jan)
* Socket reconnection fixes (Marc-André)
* icount fixes (Emilio, myself)
* QSP fixes for Coverity (myself)
* Some record/replay improovements (Pavel)
* Packed struct fixes (Peter)
* Windows dump fixes and elf2dmp (Viktor)
* kbmclock fix (Yongji)

# gpg: Signature made Tue 02 Oct 2018 18:13:12 BST
# gpg:                using RSA key BFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>"
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream: (80 commits)
  hw/scsi/mptendian: Avoid taking address of fields in packed structs
  cpus: fix TCG kick timer leak
  docs/devel/memory.txt: Document _with_attrs accessors
  hw/nvram/fw_cfg: Use memberwise copy of MemoryRegionOps struct
  memory: Remove old_mmio accessors
  memory: Fix access_with_adjusted_size(small size) on big-endian memory regions
  memory: Refactor common shifting code from accessors
  memory: Use MAKE_64BIT_MASK()
  virtio: do not take address of packed members
  replay: replay BH for IDE trim operation
  hostmem-file: make available memory-backend-file on POSIX-based hosts
  target/i386: fix translation for icount mode
  hvf: drop unused variable
  qom/object: add some interface asserts
  accel/tcg: Remove dead code
  lsi53c895a: convert to trace-events
  scsi-block: Deprecate rotation_rate
  kvmclock: run KVM_KVMCLOCK_CTRL ioctl in vcpu thread
  MAINTAINERS: add myself as elf2dmp maintainer
  contrib: add elf2dmp tool
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-10-02 18:27:18 +01:00
Peter Maydell 9786650866 hw/scsi/mptendian: Avoid taking address of fields in packed structs
Taking the address of a field in a packed struct is a bad idea, because
it might not be actually aligned enough for that pointer type (and
thus cause a crash on dereference on some host architectures). Newer
versions of clang warn about this. Avoid the bug by not using the
"modify in place" byte swapping functions.

This patch was produced with the following simple spatch script:
@@
expression E;
@@
-le16_to_cpus(&E);
+E = le16_to_cpu(E);
@@
expression E;
@@
-le32_to_cpus(&E);
+E = le32_to_cpu(E);
@@
expression E;
@@
-le64_to_cpus(&E);
+E = le64_to_cpu(E);
@@
expression E;
@@
-cpu_to_le16s(&E);
+E = cpu_to_le16(E);
@@
expression E;
@@
-cpu_to_le32s(&E);
+E = cpu_to_le32(E);
@@
expression E;
@@
-cpu_to_le64s(&E);
+E = cpu_to_le64(E);

followed by some minor tidying of overlong lines and bad indent.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20180927134852.21490-1-peter.maydell@linaro.org>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:14 +02:00
Alex Bennée 1926ab273b cpus: fix TCG kick timer leak
This is an alternative fix to Marc-André's original patch.

Reported-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20180927171724.30128-1-alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:14 +02:00
Peter Maydell 687ac05d71 docs/devel/memory.txt: Document _with_attrs accessors
When we added the _with_attrs accessors we forgot to mention
them in the documentation.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20180824170422.5783-4-peter.maydell@linaro.org>
Based-on: <20180802174042.29234-1-peter.maydell@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:14 +02:00
Peter Maydell 695e2fc2d6 hw/nvram/fw_cfg: Use memberwise copy of MemoryRegionOps struct
We've now removed the 'old_mmio' member from MemoryRegionOps,
so we can perform the copy as a simple struct copy rather
than having to do it via a memberwise copy.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20180824170422.5783-3-peter.maydell@linaro.org>
Based-on: <20180802174042.29234-1-peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:14 +02:00
Peter Maydell 62a0db942d memory: Remove old_mmio accessors
Now that all the users of old_mmio MemoryRegion accessors
have been converted, we can remove the core code support.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20180824170422.5783-2-peter.maydell@linaro.org>
Based-on: <20180802174042.29234-1-peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:14 +02:00
Philippe Mathieu-Daudé 98f52cdbb5 memory: Fix access_with_adjusted_size(small size) on big-endian memory regions
Memory regions configured as DEVICE_BIG_ENDIAN (or DEVICE_NATIVE_ENDIAN on
big-endian guest) behave incorrectly when the memory access 'size' is smaller
than the implementation 'access_size'.

In the following code segment from access_with_adjusted_size():

    if (memory_region_big_endian(mr)) {
        for (i = 0; i < size; i += access_size) {
            r |= access_fn(mr, addr + i, value, access_size,
                        (size - access_size - i) * 8, access_mask, attrs);
        }

(size - access_size - i) * 8 is the number of bits that will arithmetic
shift the current value.

Currently we can only 'left' shift a read() access, and 'right' shift a write().

When the access 'size' is smaller than the implementation, we get a negative
number of bits to shift.

For the read() case, a negative 'left' shift is a 'right' shift :)
However since the 'shift' type is unsigned, there is currently no way to
right shift.

Fix this by changing the access_fn() prototype to handle signed shift values,
and modify the memory_region_shift_read|write_access() helpers to correctly
arithmetic shift the opposite direction when the 'shift' value is negative.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180927002416.1781-4-f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:14 +02:00
Philippe Mathieu-Daudé 3c754a9383 memory: Refactor common shifting code from accessors
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180927002416.1781-3-f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:14 +02:00
Philippe Mathieu-Daudé 36960b4d66 memory: Use MAKE_64BIT_MASK()
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180927002416.1781-2-f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:13 +02:00
Paolo Bonzini d41ca5afe3 virtio: do not take address of packed members
The address of a packed member is not packed, which may cause accesses
to unaligned pointers.  Avoid this by reading the packed value before
passing it to another function.

Cc: Jason Wang <jasowang@redhat.com>
Cc: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:13 +02:00
Pavel Dovgalyuk b255df7e6e replay: replay BH for IDE trim operation
This patch makes IDE trim BH deterministic, because it affects
the device state. Therefore its invocation should be replayed
instead of running at the random moment.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20180912081950.3228.68987.stgit@pasha-VirtualBox>
Acked-by: John Snow <jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:13 +02:00
Hikaru Nishida d5dbde4645 hostmem-file: make available memory-backend-file on POSIX-based hosts
Before this change, memory-backend-file object is valid for Linux hosts
only because hostmem-file.c is compiled only on Linux hosts.
However, other POSIX-based hosts (such as macOS) can support
memory-backend-file object in the same way as on Linux hosts.
This patch makes hostmem-file.c and related functions to be compiled on
all POSIX-based hosts to make available memory-backend-file on them.

Signed-off-by: Hikaru Nishida <hikarupsp@gmail.com>
Message-Id: <20180924123205.29651-1-hikarupsp@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:13 +02:00
Pavel Dovgalyuk 41d54dc09f target/i386: fix translation for icount mode
This patch fixes the checking of boundary crossing instructions.
In icount mode only first instruction of the block may cross
the page boundary to keep the translation deterministic.
These conditions already existed, but compared the wrong variable.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Message-Id: <20180920071702.22477.43980.stgit@pasha-VirtualBox>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:13 +02:00
Paolo Bonzini 442c3b4594 hvf: drop unused variable
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:13 +02:00
Marc-André Lureau 422ca1432f qom/object: add some interface asserts
An interface can't have any instance size or callback, or itself
implement other interfaces (this is unsupported).

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180912125303.29158-1-marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:13 +02:00
Thomas Huth dcf6760a64 accel/tcg: Remove dead code
The global cpu_single_env variable has been removed more than 5 years
ago, so apparently nobody used this dead debug code in that timeframe
anymore. Thus let's remove it completely now.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <1537204134-15905-1-git-send-email-thuth@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:13 +02:00
Mark Cave-Ayland c921370b22 lsi53c895a: convert to trace-events
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Message-Id: <20180917053229.4853-1-mark.cave-ayland@ilande.co.uk>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:13 +02:00
Fam Zheng 51f43d5792 scsi-block: Deprecate rotation_rate
This option is added together with scsi-disk but is never honoured,
becuase we don't emulate the VPD page for scsi-block. We could intercept
and inject the user specified value like for max xfer len, but it's
probably not helpful since the intent of 070f80095a was for random
entropy aspects, not for performance. If emulated rotation rate is
desired, scsi-hd is more suitable.

Signed-off-by: Fam Zheng <famz@redhat.com>

Message-Id: <20180917083138.3948-1-famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:13 +02:00
Yongji Xie 4c3e250627 kvmclock: run KVM_KVMCLOCK_CTRL ioctl in vcpu thread
According to KVM API Documentation, we should only
run vcpu ioctls from the same thread that was used
to create the vcpu. This patch makes KVM_KVMCLOCK_CTRL
ioctl consistent with the Documentation.

No functional change.

Signed-off-by: Yongji Xie <xieyongji@baidu.com>
Signed-off-by: Chai Wen <chaiwen@baidu.com>
Message-Id: <1531315364-2551-1-git-send-email-xieyongji@baidu.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Yongji Xie <elohimes@gmail.com>
2018-10-02 19:09:13 +02:00
Viktor Prutyanov 5ee547bb78 MAINTAINERS: add myself as elf2dmp maintainer
Add myself as contrib/elf2dmp maintainer and elf2dmp as maintained.

Signed-off-by: Viktor Prutyanov <viktor.prutyanov@phystech.edu>
Message-Id: <20180918095422.4468-1-viktor.prutyanov@phystech.edu>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:13 +02:00
Viktor Prutyanov 3fa2d384c2 contrib: add elf2dmp tool
elf2dmp is a converter from ELF dump (produced by 'dump-guest-memory') to
Windows MEMORY.DMP format (also know as 'Complete Memory Dump') which can be
opened in WinDbg.

This tool can help if VMCoreInfo device/driver is absent in Windows VM and
'dump-guest-memory -w' is not available but dump can be created in ELF format.

The tool works as follows:
1. Determine the system paging root looking at GS_BASE or KERNEL_GS_BASE
to locate the PRCB structure and finds the kernel CR3 nearby if QEMU CPU
state CR3 is not suitable.
2. Find an address within the kernel image by dereferencing the first
IDT entry and scans virtual memory upwards until the start of the
kernel.
3. Download a PDB matching the kernel from the Microsoft symbol store,
and figure out the layout of certain relevant structures necessary for
the dump.
4. Populate the corresponding structures in the memory image and create
the appropriate dump header.

Signed-off-by: Viktor Prutyanov <viktor.prutyanov@virtuozzo.com>
Message-Id: <1535546488-30208-3-git-send-email-viktor.prutyanov@virtuozzo.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:12 +02:00
Viktor Prutyanov a52fbc37a4 dump: move Windows dump structures definitions
This patch moves definitions of Windows dump structures to
include/qemu/win_dump_defs.h to keep create_win_dump() prototype separate.

Signed-off-by: Viktor Prutyanov <viktor.prutyanov@virtuozzo.com>
Message-Id: <1535546488-30208-2-git-send-email-viktor.prutyanov@virtuozzo.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:12 +02:00
Li Qiang 8371158bba hw: edu: replace device name with macro
Just as other devices do.

Signed-off-by: Li Qiang <liq3ea@gmail.com>
Message-Id: <1536901871-2729-1-git-send-email-liq3ea@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:12 +02:00
Liran Alon 5b8063c406 i386: Compile CPUX86State xsave_buf only when support KVM or HVF
While at it, also rename var to indicate it is not used only in KVM.

Reviewed-by: Nikita Leshchenko <nikita.leshchenko@oracle.com>
Reviewed-by: Patrick Colp <patrick.colp@oracle.com>
Signed-off-by: Liran Alon <liran.alon@oracle.com>
Message-Id: <20180914003827.124570-2-liran.alon@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:12 +02:00
Paolo Bonzini f8dc4c645e target/i386: rename HF_SVMI_MASK to HF_GUEST_MASK
This flag will be used for KVM's nested VMX migration; the HF_GUEST_MASK name
is already used in KVM, adopt it in QEMU as well.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:12 +02:00
Paolo Bonzini 92d5f1a414 target/i386: unify masking of interrupts
Interrupt handling depends on various flags in env->hflags or env->hflags2,
and the exact detail were not exactly replicated between x86_cpu_has_work
and x86_cpu_exec_interrupt.  Create a new function that extracts the
highest-priority non-masked interrupt, and use it in both functions.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:12 +02:00
Paolo Bonzini 27e18b8952 char-pty: remove unnecessary #ifdef
For some reason __APPLE__ was not checked in pty code.  However, the #ifdef
is redundant: this file is already compiled only if CONFIG_POSIX, same as
util/qemu-openpty.c which it uses.

Reported-by: Roman Bolshakov <r.bolshakov@yadro.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:12 +02:00
Marc-André Lureau 5b9d17bb76 test-char: add socket reconnect test
This test exhibits a regression fixed by the previous reverts.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180817135224.22971-5-marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:12 +02:00
Marc-André Lureau 1ef64f1482 test-char: fix random socket test failure
Peter reported a test failure on FreeBSD with the new reconnect test:

MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
gtester -k --verbose -m=quick tests/test-char
TEST: tests/test-char... (pid=16190)
  /char/null:                                                          OK
  /char/invalid:                                                       OK
  /char/ringbuf:                                                       OK
  /char/mux:                                                           OK
  /char/stdio:                                                         OK
  /char/pipe:                                                          OK
  /char/file:                                                          OK
  /char/file-fifo:                                                     OK
  /char/udp:                                                           OK
  /char/serial:                                                        OK
  /char/hotswap:                                                       OK
  /char/socket/basic:                                                  OK
  /char/socket/reconnect:                                              FAIL
GTester: last random seed: R02S521380d9c12f1dac3ad1763bf5665c27
(pid=16367)
  /char/socket/fdpass:                                                 OK
FAIL: tests/test-char
**
ERROR:tests/test-char.c:353:char_socket_test_common: assertion failed:
(object_property_get_bool(OBJECT(chr_client), "connected",
&error_abort))

It turns out that the socket test code checks both server and client
connection states, but doesn't wait for both.

Wait for the client side as well.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20180823143125.16767-5-marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:12 +02:00
Marc-André Lureau dfe9ea200a char-socket: update all ioc handlers when changing context
So far, tcp_chr_update_read_handler() only updated the read
handler. Let's also update the hup handler.

Factorize the code while at it. (note that s->ioc != NULL when
s->connected)

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180817135224.22971-4-marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:11 +02:00
Marc-André Lureau 5573f98fa6 Revert "chardev: tcp: postpone async connection setup"
This reverts commit 25679e5d58.

This commit broke "reconnect socket" chardev that are created after
"machine_done": they no longer try to connect. It broke also
vhost-user-test that uses chardev while there is no "machine_done"
event.

The goal of this patch was to move the "connect" source to the
frontend context. chr->gcontext is set with
qemu_chr_fe_set_handlers(). But there is no guarantee that it will be
called, so we can't delay connection until then: the chardev should
still attempt to connect during open(). qemu_chr_fe_set_handlers() is
eventually called later and will update the context.

Unless there is a good reason to not use initially the default
context, I think we should revert to the previous state to fix the
regressions.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180817135224.22971-3-marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-10-02 19:09:11 +02:00