Discussion:
Are there some protective mechanisms of valgrind on MIPS64, especially on Cavium OCTEON3?
(too old to reply)
网络尖兵
2017-04-18 09:21:33 UTC
Permalink
Raw Message
Hello everyone,
I am trying to use valgrind's memchek tool to find out a segment fault problem. The user space program was coredumped every time when it was running without valgrind. But when it was running with valgrind, very strangely, it was not coredumped! Then I wrote a script to give some pressure to the program, and it run smoothly for several hours.


So I am confusing and writing this mail to find out if there are some protective mechanisms of valgrind on MIPS64, especially on Cavium OCTEON3?
Petar Jovanovic
2017-04-18 11:00:35 UTC
Permalink
Raw Message
Post by 网络尖兵
Hello everyone,
I am trying to use valgrind's memchek tool to find out a segment fault
problem. The user space program was coredumped every time when it was
running without valgrind. But when it was running with valgrind, very
strangely, it was not coredumped! Then I wrote a script to give some
pressure to the program, and it run smoothly for several hours.
So I am confusing and writing this mail to find out if there are some
protective mechanisms of valgrind on MIPS64, especially on Cavium OCTEON3?
Can you share some information about the actual segfault (when running
without Valgrind)? What is causing the failure?
zboom
2017-04-19 01:51:52 UTC
Permalink
Raw Message
Thanks for your reply!
This problem occurred when I was running OpenSwitch
(http://www.openswitch.net/) on my embedded system. As you know, the
OpenSwitch use OVS (http://www.openvswitch.org/) as its control-plane. And
It have much more configurations than OVS. For example, there are more than
one hundred interfaces. When the ovsdb-client monitored this kind of
messages, the ovsdb-server will return a huge JSON message (about 24KB).
This huge message could cause the EAGAIN for several times, and then the
ovsdb-server would be coredumped. Because there were two recursive
functions, first I doubted the stack was overflowed. But I have confirmed
with the ovs-dev group, and have counted the recursive depth. The result I
got was safe for the stack size.

The following are two typical call trace, and the ovsdb-server program
coredumped at very different place every time.
1.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x000000fff464a154 in hmap_remove (node=0xfff36e89c0, hmap=0xfff36313c0)
at /usr/src/debug/ops-openvswitch/git999-r0/lib/hmap.h:245
245 /usr/src/debug/ops-openvswitch/git999-r0/lib/hmap.h: No such file or
directory.
(gdb) bt
#0 0x000000fff464a154 in hmap_remove (node=0xfff36e89c0, hmap=0xfff36313c0)
at /usr/src/debug/ops-openvswitch/git999-r0/lib/hmap.h:245
#1 shash_steal (sh=0xfff36313c0, node=0xfff36e89c0) at
/usr/src/debug/ops-openvswitch/git999-r0/lib/shash.c:187
#2 0x000000fff464a1cc in shash_delete (sh=<optimized out>, node=<optimized
out>) at /usr/src/debug/ops-openvswitch/git999-r0/lib/shash.c:176
#3 0x000000fff463ae34 in json_destroy_object (object=0xfff36313c0) at
/usr/src/debug/ops-openvswitch/git999-r0/lib/json.c:386
#4 json_destroy (json=0xfff36313a0) at
/usr/src/debug/ops-openvswitch/git999-r0/lib/json.c:352
#5 0x000000fff463ae24 in json_destroy_object (object=0xfff38e9420) at
/usr/src/debug/ops-openvswitch/git999-r0/lib/json.c:385
#6 json_destroy (json=0xfff38e9400) at
/usr/src/debug/ops-openvswitch/git999-r0/lib/json.c:352
#7 0x000000fff463ae24 in json_destroy_object (object=0xfff38cd600) at
/usr/src/debug/ops-openvswitch/git999-r0/lib/json.c:385
#8 json_destroy (json=0xfff38cd620) at
/usr/src/debug/ops-openvswitch/git999-r0/lib/json.c:352
#9 0x000000fff465e100 in jsonrpc_send (rpc=0xfff3832680, msg=<optimized
out>) at /usr/src/debug/ops-openvswitch/git999-r0/lib/jsonrpc.c:261
#10 0x000000fff465ed30 in jsonrpc_session_send (s=<optimized out>,
msg=<optimized out>)
at /usr/src/debug/ops-openvswitch/git999-r0/lib/jsonrpc.c:1029
#11 0x000000fff43ee87c in ovsdb_jsonrpc_session_send
(s=***@entry=0xfff3619e00, msg=0xfff36dd170)
at /usr/src/debug/ops-openvswitch/git999-r0/ovsdb/jsonrpc-server.c:1290
#12 0x000000fff43f2ac4 in ovsdb_jsonrpc_session_got_request
(request=0xfff381cbd0, s=<optimized out>)
at /usr/src/debug/ops-openvswitch/git999-r0/ovsdb/jsonrpc-server.c:1256
#13 ovsdb_jsonrpc_session_run (s=<optimized out>) at
/usr/src/debug/ops-openvswitch/git999-r0/ovsdb/jsonrpc-server.c:523
#14 ovsdb_jsonrpc_session_run_all (remote=<optimized out>) at
/usr/src/debug/ops-openvswitch/git999-r0/ovsdb/jsonrpc-server.c:562
#15 ovsdb_jsonrpc_server_run (svr=0x0) at
/usr/src/debug/ops-openvswitch/git999-r0/ovsdb/jsonrpc-server.c:366
#16 0x0000000120005d60 in main_loop (exiting=0xffdfaaf8d0, run_process=0x0,
remotes=0xffdfaaf888, unixctl=0x12010e4a0, all_dbs=0xffdfaaf818,
jsonrpc=0x0) at
/usr/src/debug/ops-openvswitch/git999-r0/ovsdb/ovsdb-server.c:161
#17 main (argc=1, argv=0x12fc19e90) at
/usr/src/debug/ops-openvswitch/git999-r0/ovsdb/ovsdb-server.c:357

(gdb) bt
#0 json_serialize (json=0x3fd28f747, s=***@entry=0xffdfecf5a0) at
/usr/src/debug/ops-openvswitch/git999-r0/lib/json.c:1499
#1 0x000000ffe999a024 in json_serialize_array (s=0xffdfecf5a0,
array=0xffe8818ec8)
at /usr/src/debug/ops-openvswitch/git999-r0/lib/json.c:1617
#2 json_serialize (json=0xffe8818ec0, s=***@entry=0xffdfecf5a0) at
/usr/src/debug/ops-openvswitch/git999-r0/lib/json.c:1517
#3 0x000000ffe9999c80 in json_serialize_object_member (i=***@entry=0,
node=***@entry=0xffe8818d60, s=***@entry=0xffdfecf5a0)
at /usr/src/debug/ops-openvswitch/git999-r0/lib/json.c:1563
#4 0x000000ffe9999e60 in json_serialize_object (s=0xffdfecf5a0,
object=0xffe8818de0)
at /usr/src/debug/ops-openvswitch/git999-r0/lib/json.c:1592
#5 json_serialize (json=<optimized out>, s=***@entry=0xffdfecf5a0)
at /usr/src/debug/ops-openvswitch/git999-r0/lib/json.c:1513
#6 0x000000ffe999d210 in json_to_ds (json=<optimized out>, flags=<optimized
out>, ds=<optimized out>)
at /usr/src/debug/ops-openvswitch/git999-r0/lib/json.c:1491
#7 0x000000ffe99bde70 in jsonrpc_send (rpc=0xffe8c32680, msg=<optimized
out>)
at /usr/src/debug/ops-openvswitch/git999-r0/lib/jsonrpc.c:252
#8 0x000000ffe99be9a8 in jsonrpc_session_send (s=<optimized out>,
msg=<optimized out>)
at /usr/src/debug/ops-openvswitch/git999-r0/lib/jsonrpc.c:1013
#9 0x000000ffe974e87c in ovsdb_jsonrpc_session_send
(s=***@entry=0xffe89b8600, msg=0xffe8a85060)
at /usr/src/debug/ops-openvswitch/git999-r0/ovsdb/jsonrpc-server.c:1290
#10 0x000000ffe9752ac4 in ovsdb_jsonrpc_session_got_request
(request=0xffe8a85150, s=<optimized out>)
at /usr/src/debug/ops-openvswitch/git999-r0/ovsdb/jsonrpc-server.c:1256
#11 ovsdb_jsonrpc_session_run (s=<optimized out>) at
/usr/src/debug/ops-openvswitch/git999-r0/ovsdb/jsonrpc-server.c:523
#12 ovsdb_jsonrpc_session_run_all (remote=<optimized out>)
at /usr/src/debug/ops-openvswitch/git999-r0/ovsdb/jsonrpc-server.c:562
#13 ovsdb_jsonrpc_server_run (svr=0x0) at
/usr/src/debug/ops-openvswitch/git999-r0/ovsdb/jsonrpc-server.c:366
#14 0x0000000120005d3c in main_loop (exiting=0xffdfecf8b0, run_process=0x0,
remotes=0xffdfecf868, unixctl=0x12010e4a0,
all_dbs=0xffdfecf7f8, jsonrpc=0x0) at
/usr/src/debug/ops-openvswitch/git999-r0/ovsdb/ovsdb-server.c:161
#15 main (argc=1, argv=0x12cf24060) at
/usr/src/debug/ops-openvswitch/git999-r0/ovsdb/ovsdb-server.c:357



--
View this message in context: http://valgrind.10908.n7.nabble.com/Are-there-some-protective-mechanisms-of-valgrind-on-MIPS64-especially-on-Cavium-OCTEON3-tp57593p57596.html
Sent from the Valgrind - Dev mailing list archive at Nabble.com.
Petar Jovanovic
2017-04-20 10:56:33 UTC
Permalink
Raw Message
Post by zboom
Thanks for your reply!
This problem occurred when I was running OpenSwitch
(http://www.openswitch.net/) on my embedded system.
Has Valgrind reported any errors in your program?
If the program has a bug, you may want to fix that first. The issue you
are reporting could be, for instance, due to reads in a freed block or
branches based on uninitialized values and such.

Petar
网络尖兵
2017-04-23 13:57:21 UTC
Permalink
Raw Message
The Valgrind didn't reported any errors in my program. That is why I asked the question before. The most strange thing is that it did not core dumped when it was running with valgrind. I agree with you, there might be a wild access to memory. But why it disappeared when it was running with valgrind?






------------------ Ô­ÊŒÓÊŒþ ------------------
·¢ŒþÈË: "Petar Jovanovic";<***@gmail.com>;
·¢ËÍʱŒä: 2017Äê4ÔÂ20ÈÕ(ÐÇÆÚËÄ) ÍíÉÏ6:56
ÊÕŒþÈË: "ÍøÂçŒâ±ø"<***@foxmail.com>;
³­ËÍ: "Valgrind Developers"<valgrind-***@lists.sourceforge.net>;
Ö÷Ìâ: Re: [Valgrind-developers] Are there some protective mechanisms ofvalgrind on MIPS64, especially on Cavium OCTEON3?
Post by zboom
Thanks for your reply!
This problem occurred when I was running OpenSwitch
(http://www.openswitch.net/) on my embedded system.
Has Valgrind reported any errors in your program?
If the program has a bug, you may want to fix that first. The issue you
are reporting could be, for instance, due to reads in a freed block or
branches based on uninitialized values and such.

Petar
Petar Jovanovic
2017-04-24 17:50:16 UTC
Permalink
Raw Message
Post by 网络尖兵
The Valgrind didn't reported any errors in my program. That is why I asked
Have you run all tools that come with Valgrind? Try different tools.
Different time or network related events could also be the cause of the
behaviour you are seeing when your program is run under Valgrind.
Loading...