← Back to team overview

maria-developers team mailing list archive

Re: Crash during shutdown (need help from PBXT team)

 

Hi Kristian & Vlad,

The crash also occurs immediately if MariaDB is started with an unknown command line parameter.

In this case the server shuts down on startup, and new_thd- >variables.table_plugin is NULL as described below.

On Dec 7, 2009, at 1:34 PM, Kristian Nielsen wrote:

Vladimir Kolesnikov <vladimir@xxxxxxxxxxxxx> writes:

Unfortunately I cannot repeat the problem on my machine. Can you
please put the branch to the debian box that you gave earlier access
to?

Yes.

There is now a build directory on the box:

   /home/buildbot/shutdown-crash/mysql-5.1.41-MariaDB-beta/

(I pushed the corresponding bzr version to
lp:~knielsen/maria/maria-pbxt-shutdown-crash. This is a merge of PBXT into
MariaDB).

I built using

   BUILD/compile-pentium-max

I get the test failure repeatably with this:

   (cd mysql-test && perl mysql-test-run.pl mysqld_option_err)

It does not seem to fail the same way every time, but it does seem to fail
everytime I run it.

Let me know if you need anything else.

Thanks again for looking into this!

- Kristian.

Kristian Nielsen wrote:
[I Cc'ed Paul, let me know if there is another prefered way of communicating,
like using Launchpad bugs or some mailing list]

Vladimir Kolesnikov <vladimir@xxxxxxxxxxxxx> writes:


"Crash during shutdown (need help from PBXT team)"

can you provide any details?


Yes, of course! Just been busy with other stuff, but very happy that you
mention it on your own initiative!

Basically, after merging PBXT I see crashes in the main.mysqld_option_err test
case. Here is an example build from Buildbot showing this:

   http://askmonty.org/buildbot/builders/debian5-i386-fulltest/builds/52
   http://askmonty.org/buildbot/builders/debian5-i386-fulltest/builds/52/steps/test_4/logs/stdio

The failure appears to be somewhat random, but I was able to repeat it easily enough locally using a release build. Debug build did not seem to show the
problem.

The issue is with the fix that Paul pushed for Bug#489088:

   https://bugs.launchpad.net/pbxt/+bug/489088

=== modified file 'storage/pbxt/src/myxt_xt.cc'
--- storage/pbxt/src/myxt_xt.cc	2009-11-27 15:37:02 +0000
+++ storage/pbxt/src/myxt_xt.cc	2009-12-01 09:50:46 +0000
@@ -3041,6 +3041,14 @@ xtPublic MX_CHARSET_INFO *myxt_getcharse
	return (MX_CHARSET_INFO *)&my_charset_utf8_general_ci;
}
+#ifdef DBUG_OFF
+//typedef struct st_plugin_int *plugin_ref;
+#define REF_MYSQL_PLUGIN(x)		(x)
+#else
+//typedef struct st_plugin_int **plugin_ref;
+#define REF_MYSQL_PLUGIN(x)		(*(x))
+#endif
+
xtPublic void *myxt_create_thread()
{
#ifdef DRIZZLED

+ LEX_STRING& plugin_name = REF_MYSQL_PLUGIN(new_thd- >variables.table_plugin)->name; + if ((plugin_name.length == 4) && (strncmp(plugin_name.str, "PBXT", plugin_name.length) == 0)) {
+		REF_MYSQL_PLUGIN(new_thd->variables.table_plugin)->ref_count--;
+	}
	new_thd->thread_stack = (char *) &new_thd;
	new_thd->store_globals();
	lex_start(new_thd);

This code crashes because new_thd->variables.table_plugin is NULL at this
point in the code (or at least sometimes is).

So this problem is similar to the previous problem handled just above in the
code with global_system_variables.table_plugin being NULL.

I tried a patch like this:

=== modified file 'storage/pbxt/src/myxt_xt.cc'
--- storage/pbxt/src/myxt_xt.cc	2009-12-01 09:50:46 +0000
+++ storage/pbxt/src/myxt_xt.cc	2009-12-04 21:41:25 +0000
@@ -3112,10 +3112,15 @@ xtPublic void *myxt_create_thread()
	 * references to the PBXT plugin object and will effectively
deadlock the plugin so  	 * that server will have to force
plugin shutdown. To avoid deadlocking and forced shutdown  	 * we
must dereference the plugin after creating THD objects.
+ * Similarly to global_system_variables.table_plugin as described above, + * new_thd->valriables.table_plugin can also become NULL due to shutdown.
	 */
- LEX_STRING& plugin_name = REF_MYSQL_PLUGIN(new_thd- >variables.table_plugin)->name; - if ((plugin_name.length == 4) && (strncmp(plugin_name.str, "PBXT", plugin_name.length) == 0)) {
-		REF_MYSQL_PLUGIN(new_thd->variables.table_plugin)->ref_count--;
+	plugin_ref table_plugin = new_thd->variables.table_plugin;
+	if (table_plugin) {
+		LEX_STRING& plugin_name = REF_MYSQL_PLUGIN(table_plugin)->name;
+ if ((plugin_name.length == 4) && (strncmp(plugin_name.str, "PBXT", plugin_name.length) == 0)) {
+			REF_MYSQL_PLUGIN(table_plugin)->ref_count--;
+		}
	}
	new_thd->thread_stack = (char *) &new_thd;
	new_thd->store_globals();

So this fixes the crashes, but of course re-introduces the original problem in
Bug#489088 with warnings about forced shutdown.

I think you should be able to repeat the problem easily enough on Linux with a non-debug build running the test main.mysqld_option_err, but if not let me
know and I will help set something up.

- Kristian.



_______________________________________________
Mailing list: https://launchpad.net/~maria-developers
Post to     : maria-developers@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list: https://launchpad.net/~maria-developers
Post to     : maria-developers@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp



--
Paul McCullagh
PrimeBase Technologies
www.primebase.org
www.blobstreaming.org
pbxt.blogspot.com






References