mirror of https://github.com/PCSX2/pcsx2.git
3037 lines
85 KiB
Plaintext
3037 lines
85 KiB
Plaintext
README.CV -- Condition Variables
|
|
--------------------------------
|
|
|
|
The original implementation of condition variables in
|
|
pthreads-win32 was based on a discussion paper:
|
|
|
|
"Strategies for Implementing POSIX Condition Variables
|
|
on Win32": http://www.cs.wustl.edu/~schmidt/win32-cv-1.html
|
|
|
|
The changes suggested below were made on Feb 6 2001. This
|
|
file is included in the package for the benefit of anyone
|
|
interested in understanding the pthreads-win32 implementation
|
|
of condition variables and the (sometimes subtle) issues that
|
|
it attempts to resolve.
|
|
|
|
Thanks go to the individuals whose names appear throughout
|
|
the following text.
|
|
|
|
Ross Johnson
|
|
|
|
--------------------------------------------------------------------
|
|
|
|
fyi.. (more detailed problem description/demos + possible fix/patch)
|
|
|
|
regards,
|
|
alexander.
|
|
|
|
|
|
Alexander Terekhov
|
|
31.01.2001 17:43
|
|
|
|
To: ace-bugs@cs.wustl.edu
|
|
cc:
|
|
From: Alexander Terekhov/Germany/IBM@IBMDE
|
|
Subject: Implementation of POSIX CVs: spur.wakeups/lost
|
|
signals/deadlocks/unfairness
|
|
|
|
|
|
|
|
ACE VERSION:
|
|
|
|
5.1.12 (pthread-win32 snapshot 2000-12-29)
|
|
|
|
HOST MACHINE and OPERATING SYSTEM:
|
|
|
|
IBM IntelliStation Z Pro, 2 x XEON 1GHz, Win2K
|
|
|
|
TARGET MACHINE and OPERATING SYSTEM, if different from HOST:
|
|
COMPILER NAME AND VERSION (AND PATCHLEVEL):
|
|
|
|
Microsoft Visual C++ 6.0
|
|
|
|
AREA/CLASS/EXAMPLE AFFECTED:
|
|
|
|
Implementation of POSIX condition variables - OS.cpp/.h
|
|
|
|
DOES THE PROBLEM AFFECT:
|
|
|
|
EXECUTION? YES!
|
|
|
|
SYNOPSIS:
|
|
|
|
a) spurious wakeups (minor problem)
|
|
b) lost signals
|
|
c) broadcast deadlock
|
|
d) unfairness (minor problem)
|
|
|
|
DESCRIPTION:
|
|
|
|
Please see attached copy of discussion thread
|
|
from comp.programming.threads for more details on
|
|
some reported problems. (i've also posted a "fyi"
|
|
message to ace-users a week or two ago but
|
|
unfortunately did not get any response so far).
|
|
|
|
It seems that current implementation suffers from
|
|
two essential problems:
|
|
|
|
1) cond.waiters_count does not accurately reflect
|
|
number of waiters blocked on semaphore - w/o
|
|
proper synchronisation that could result (in the
|
|
time window when counter is not accurate)
|
|
in spurious wakeups organised by subsequent
|
|
_signals and _broadcasts.
|
|
|
|
2) Always having (with no e.g. copy_and_clear/..)
|
|
the same queue in use (semaphore+counter)
|
|
neither signal nor broadcast provide 'atomic'
|
|
behaviour with respect to other threads/subsequent
|
|
calls to signal/broadcast/wait.
|
|
|
|
Each problem and combination of both could produce
|
|
various nasty things:
|
|
|
|
a) spurious wakeups (minor problem)
|
|
|
|
it is possible that waiter(s) which was already
|
|
unblocked even so is still counted as blocked
|
|
waiter. signal and broadcast will release
|
|
semaphore which will produce a spurious wakeup
|
|
for a 'real' waiter coming later.
|
|
|
|
b) lost signals
|
|
|
|
signalling thread ends up consuming its own
|
|
signal. please see demo/discussion below.
|
|
|
|
c) broadcast deadlock
|
|
|
|
last_waiter processing code does not correctly
|
|
handle the case with multiple threads
|
|
waiting for the end of broadcast.
|
|
please see demo/discussion below.
|
|
|
|
d) unfairness (minor problem)
|
|
|
|
without SignalObjectAndWait some waiter(s)
|
|
may end up consuming broadcasted signals
|
|
multiple times (spurious wakeups) because waiter
|
|
thread(s) can be preempted before they call
|
|
semaphore wait (but after count++ and mtx.unlock).
|
|
|
|
REPEAT BY:
|
|
|
|
See below... run problem demos programs (tennis.cpp and
|
|
tennisb.cpp) number of times concurrently (on multiprocessor)
|
|
and in multiple sessions or just add a couple of "Sleep"s
|
|
as described in the attached copy of discussion thread
|
|
from comp.programming.threads
|
|
|
|
SAMPLE FIX/WORKAROUND:
|
|
|
|
See attached patch to pthread-win32.. well, I can not
|
|
claim that it is completely bug free but at least my
|
|
test and tests provided by pthreads-win32 seem to work.
|
|
Perhaps that will help.
|
|
|
|
regards,
|
|
alexander.
|
|
|
|
|
|
>> Forum: comp.programming.threads
|
|
>> Thread: pthread_cond_* implementation questions
|
|
.
|
|
.
|
|
.
|
|
David Schwartz <davids@webmaster.com> wrote:
|
|
|
|
> terekhov@my-deja.com wrote:
|
|
>
|
|
>> BTW, could you please also share your view on other perceived
|
|
>> "problems" such as nested broadcast deadlock, spurious wakeups
|
|
>> and (the latest one) lost signals??
|
|
>
|
|
>I'm not sure what you mean. The standard allows an implementation
|
|
>to do almost whatever it likes. In fact, you could implement
|
|
>pthread_cond_wait by releasing the mutex, sleeping a random
|
|
>amount of time, and then reacquiring the mutex. Of course,
|
|
>this would be a pretty poor implementation, but any code that
|
|
>didn't work under that implementation wouldn't be strictly
|
|
>compliant.
|
|
|
|
The implementation you suggested is indeed correct
|
|
one (yes, now I see it :). However it requires from
|
|
signal/broadcast nothing more than to "{ return 0; }"
|
|
That is not the case for pthread-win32 and ACE
|
|
implementations. I do think that these implementations
|
|
(basically the same implementation) have some serious
|
|
problems with wait/signal/broadcast calls. I am looking
|
|
for help to clarify whether these problems are real
|
|
or not. I think that I can demonstrate what I mean
|
|
using one or two small sample programs.
|
|
.
|
|
.
|
|
.
|
|
==========
|
|
tennis.cpp
|
|
==========
|
|
|
|
#include "ace/Synch.h"
|
|
#include "ace/Thread.h"
|
|
|
|
enum GAME_STATE {
|
|
|
|
START_GAME,
|
|
PLAYER_A, // Player A playes the ball
|
|
PLAYER_B, // Player B playes the ball
|
|
GAME_OVER,
|
|
ONE_PLAYER_GONE,
|
|
BOTH_PLAYERS_GONE
|
|
|
|
};
|
|
|
|
enum GAME_STATE eGameState;
|
|
ACE_Mutex* pmtxGameStateLock;
|
|
ACE_Condition< ACE_Mutex >* pcndGameStateChange;
|
|
|
|
void*
|
|
playerA(
|
|
void* pParm
|
|
)
|
|
{
|
|
|
|
// For access to game state variable
|
|
pmtxGameStateLock->acquire();
|
|
|
|
// Play loop
|
|
while ( eGameState < GAME_OVER ) {
|
|
|
|
// Play the ball
|
|
cout << endl << "PLAYER-A" << endl;
|
|
|
|
// Now its PLAYER-B's turn
|
|
eGameState = PLAYER_B;
|
|
|
|
// Signal to PLAYER-B that now it is his turn
|
|
pcndGameStateChange->signal();
|
|
|
|
// Wait until PLAYER-B finishes playing the ball
|
|
do {
|
|
|
|
pcndGameStateChange->wait();
|
|
|
|
if ( PLAYER_B == eGameState )
|
|
cout << endl << "----PLAYER-A: SPURIOUS WAKEUP!!!" << endl;
|
|
|
|
} while ( PLAYER_B == eGameState );
|
|
|
|
}
|
|
|
|
// PLAYER-A gone
|
|
eGameState = (GAME_STATE)(eGameState+1);
|
|
cout << endl << "PLAYER-A GONE" << endl;
|
|
|
|
// No more access to state variable needed
|
|
pmtxGameStateLock->release();
|
|
|
|
// Signal PLAYER-A gone event
|
|
pcndGameStateChange->broadcast();
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
void*
|
|
playerB(
|
|
void* pParm
|
|
)
|
|
{
|
|
|
|
// For access to game state variable
|
|
pmtxGameStateLock->acquire();
|
|
|
|
// Play loop
|
|
while ( eGameState < GAME_OVER ) {
|
|
|
|
// Play the ball
|
|
cout << endl << "PLAYER-B" << endl;
|
|
|
|
// Now its PLAYER-A's turn
|
|
eGameState = PLAYER_A;
|
|
|
|
// Signal to PLAYER-A that now it is his turn
|
|
pcndGameStateChange->signal();
|
|
|
|
// Wait until PLAYER-A finishes playing the ball
|
|
do {
|
|
|
|
pcndGameStateChange->wait();
|
|
|
|
if ( PLAYER_A == eGameState )
|
|
cout << endl << "----PLAYER-B: SPURIOUS WAKEUP!!!" << endl;
|
|
|
|
} while ( PLAYER_A == eGameState );
|
|
|
|
}
|
|
|
|
// PLAYER-B gone
|
|
eGameState = (GAME_STATE)(eGameState+1);
|
|
cout << endl << "PLAYER-B GONE" << endl;
|
|
|
|
// No more access to state variable needed
|
|
pmtxGameStateLock->release();
|
|
|
|
// Signal PLAYER-B gone event
|
|
pcndGameStateChange->broadcast();
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
int
|
|
main (int, ACE_TCHAR *[])
|
|
{
|
|
|
|
pmtxGameStateLock = new ACE_Mutex();
|
|
pcndGameStateChange = new ACE_Condition< ACE_Mutex >( *pmtxGameStateLock
|
|
);
|
|
|
|
// Set initial state
|
|
eGameState = START_GAME;
|
|
|
|
// Create players
|
|
ACE_Thread::spawn( playerA );
|
|
ACE_Thread::spawn( playerB );
|
|
|
|
// Give them 5 sec. to play
|
|
Sleep( 5000 );//sleep( 5 );
|
|
|
|
// Set game over state
|
|
pmtxGameStateLock->acquire();
|
|
eGameState = GAME_OVER;
|
|
|
|
// Let them know
|
|
pcndGameStateChange->broadcast();
|
|
|
|
// Wait for players to stop
|
|
do {
|
|
|
|
pcndGameStateChange->wait();
|
|
|
|
} while ( eGameState < BOTH_PLAYERS_GONE );
|
|
|
|
// Cleanup
|
|
cout << endl << "GAME OVER" << endl;
|
|
pmtxGameStateLock->release();
|
|
delete pcndGameStateChange;
|
|
delete pmtxGameStateLock;
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
===========
|
|
tennisb.cpp
|
|
===========
|
|
#include "ace/Synch.h"
|
|
#include "ace/Thread.h"
|
|
|
|
enum GAME_STATE {
|
|
|
|
START_GAME,
|
|
PLAYER_A, // Player A playes the ball
|
|
PLAYER_B, // Player B playes the ball
|
|
GAME_OVER,
|
|
ONE_PLAYER_GONE,
|
|
BOTH_PLAYERS_GONE
|
|
|
|
};
|
|
|
|
enum GAME_STATE eGameState;
|
|
ACE_Mutex* pmtxGameStateLock;
|
|
ACE_Condition< ACE_Mutex >* pcndGameStateChange;
|
|
|
|
void*
|
|
playerA(
|
|
void* pParm
|
|
)
|
|
{
|
|
|
|
// For access to game state variable
|
|
pmtxGameStateLock->acquire();
|
|
|
|
// Play loop
|
|
while ( eGameState < GAME_OVER ) {
|
|
|
|
// Play the ball
|
|
cout << endl << "PLAYER-A" << endl;
|
|
|
|
// Now its PLAYER-B's turn
|
|
eGameState = PLAYER_B;
|
|
|
|
// Signal to PLAYER-B that now it is his turn
|
|
pcndGameStateChange->broadcast();
|
|
|
|
// Wait until PLAYER-B finishes playing the ball
|
|
do {
|
|
|
|
pcndGameStateChange->wait();
|
|
|
|
if ( PLAYER_B == eGameState )
|
|
cout << endl << "----PLAYER-A: SPURIOUS WAKEUP!!!" << endl;
|
|
|
|
} while ( PLAYER_B == eGameState );
|
|
|
|
}
|
|
|
|
// PLAYER-A gone
|
|
eGameState = (GAME_STATE)(eGameState+1);
|
|
cout << endl << "PLAYER-A GONE" << endl;
|
|
|
|
// No more access to state variable needed
|
|
pmtxGameStateLock->release();
|
|
|
|
// Signal PLAYER-A gone event
|
|
pcndGameStateChange->broadcast();
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
void*
|
|
playerB(
|
|
void* pParm
|
|
)
|
|
{
|
|
|
|
// For access to game state variable
|
|
pmtxGameStateLock->acquire();
|
|
|
|
// Play loop
|
|
while ( eGameState < GAME_OVER ) {
|
|
|
|
// Play the ball
|
|
cout << endl << "PLAYER-B" << endl;
|
|
|
|
// Now its PLAYER-A's turn
|
|
eGameState = PLAYER_A;
|
|
|
|
// Signal to PLAYER-A that now it is his turn
|
|
pcndGameStateChange->broadcast();
|
|
|
|
// Wait until PLAYER-A finishes playing the ball
|
|
do {
|
|
|
|
pcndGameStateChange->wait();
|
|
|
|
if ( PLAYER_A == eGameState )
|
|
cout << endl << "----PLAYER-B: SPURIOUS WAKEUP!!!" << endl;
|
|
|
|
} while ( PLAYER_A == eGameState );
|
|
|
|
}
|
|
|
|
// PLAYER-B gone
|
|
eGameState = (GAME_STATE)(eGameState+1);
|
|
cout << endl << "PLAYER-B GONE" << endl;
|
|
|
|
// No more access to state variable needed
|
|
pmtxGameStateLock->release();
|
|
|
|
// Signal PLAYER-B gone event
|
|
pcndGameStateChange->broadcast();
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
int
|
|
main (int, ACE_TCHAR *[])
|
|
{
|
|
|
|
pmtxGameStateLock = new ACE_Mutex();
|
|
pcndGameStateChange = new ACE_Condition< ACE_Mutex >( *pmtxGameStateLock
|
|
);
|
|
|
|
// Set initial state
|
|
eGameState = START_GAME;
|
|
|
|
// Create players
|
|
ACE_Thread::spawn( playerA );
|
|
ACE_Thread::spawn( playerB );
|
|
|
|
// Give them 5 sec. to play
|
|
Sleep( 5000 );//sleep( 5 );
|
|
|
|
// Make some noise
|
|
pmtxGameStateLock->acquire();
|
|
cout << endl << "---Noise ON..." << endl;
|
|
pmtxGameStateLock->release();
|
|
for ( int i = 0; i < 100000; i++ )
|
|
pcndGameStateChange->broadcast();
|
|
cout << endl << "---Noise OFF" << endl;
|
|
|
|
// Set game over state
|
|
pmtxGameStateLock->acquire();
|
|
eGameState = GAME_OVER;
|
|
cout << endl << "---Stopping the game..." << endl;
|
|
|
|
// Let them know
|
|
pcndGameStateChange->broadcast();
|
|
|
|
// Wait for players to stop
|
|
do {
|
|
|
|
pcndGameStateChange->wait();
|
|
|
|
} while ( eGameState < BOTH_PLAYERS_GONE );
|
|
|
|
// Cleanup
|
|
cout << endl << "GAME OVER" << endl;
|
|
pmtxGameStateLock->release();
|
|
delete pcndGameStateChange;
|
|
delete pmtxGameStateLock;
|
|
|
|
return 0;
|
|
|
|
}
|
|
.
|
|
.
|
|
.
|
|
David Schwartz <davids@webmaster.com> wrote:
|
|
>> > It's compliant
|
|
>>
|
|
>> That is really good.
|
|
>
|
|
>> Tomorrow (I have to go urgently now) I will try to
|
|
>> demonstrate the lost-signal "problem" of current
|
|
>> pthread-win32 and ACE-(variant w/o SingleObjectAndWait)
|
|
>> implementations: players start suddenly drop their balls :-)
|
|
>> (with no change in source code).
|
|
>
|
|
>Signals aren't lost, they're going to the main thread,
|
|
>which isn't coded correctly to handle them. Try this:
|
|
>
|
|
> // Wait for players to stop
|
|
> do {
|
|
>
|
|
> pthread_cond_wait( &cndGameStateChange,&mtxGameStateLock );
|
|
>printf("Main thread stole a signal\n");
|
|
>
|
|
> } while ( eGameState < BOTH_PLAYERS_GONE );
|
|
>
|
|
>I bet everytime you thing a signal is lost, you'll see that printf.
|
|
>The signal isn't lost, it was stolen by another thread.
|
|
|
|
well, you can probably loose your bet.. it was indeed stolen
|
|
by "another" thread but not the one you seem to think of.
|
|
|
|
I think that what actually happens is the following:
|
|
|
|
H:\SA\UXX\pt\PTHREADS\TESTS>tennis3.exe
|
|
|
|
PLAYER-A
|
|
|
|
PLAYER-B
|
|
|
|
----PLAYER-B: SPURIOUS WAKEUP!!!
|
|
|
|
PLAYER-A GONE
|
|
|
|
PLAYER-B GONE
|
|
|
|
GAME OVER
|
|
|
|
H:\SA\UXX\pt\PTHREADS\TESTS>
|
|
|
|
here you can see that PLAYER-B after playing his first
|
|
ball (which came via signal from PLAYER-A) just dropped
|
|
it down. What happened is that his signal to player A
|
|
was consumed as spurious wakeup by himself (player B).
|
|
|
|
The implementation has a problem:
|
|
|
|
================
|
|
waiting threads:
|
|
================
|
|
|
|
{ /** Critical Section
|
|
|
|
inc cond.waiters_count
|
|
|
|
}
|
|
|
|
/*
|
|
/* Atomic only if using Win32 SignalObjectAndWait
|
|
/*
|
|
cond.mtx.release
|
|
|
|
/*** ^^-- A THREAD WHICH DID SIGNAL MAY ACQUIRE THE MUTEX,
|
|
/*** GO INTO WAIT ON THE SAME CONDITION AND OVERTAKE
|
|
/*** ORIGINAL WAITER(S) CONSUMING ITS OWN SIGNAL!
|
|
|
|
cond.sem.wait
|
|
|
|
Player-A after playing game's initial ball went into
|
|
wait (called _wait) but was pre-empted before reaching
|
|
wait semaphore. He was counted as waiter but was not
|
|
actually waiting/blocked yet.
|
|
|
|
===============
|
|
signal threads:
|
|
===============
|
|
|
|
{ /** Critical Section
|
|
|
|
waiters_count = cond.waiters_count
|
|
|
|
}
|
|
|
|
if ( waiters_count != 0 )
|
|
|
|
sem.post 1
|
|
|
|
endif
|
|
|
|
Player-B after he received signal/ball from Player A
|
|
called _signal. The _signal did see that there was
|
|
one waiter blocked on the condition (Player-A) and
|
|
released the semaphore.. (but it did not unblock
|
|
Player-A because he was not actually blocked).
|
|
Player-B thread continued its execution, called _wait,
|
|
was counted as second waiter BUT was allowed to slip
|
|
through opened semaphore gate (which was opened for
|
|
Player-B) and received his own signal. Player B remained
|
|
blocked followed by Player A. Deadlock happened which
|
|
lasted until main thread came in and said game over.
|
|
|
|
It seems to me that the implementation fails to
|
|
correctly implement the following statement
|
|
from specification:
|
|
|
|
http://www.opengroup.org/
|
|
onlinepubs/007908799/xsh/pthread_cond_wait.html
|
|
|
|
"These functions atomically release mutex and cause
|
|
the calling thread to block on the condition variable
|
|
cond; atomically here means "atomically with respect
|
|
to access by another thread to the mutex and then the
|
|
condition variable". That is, if another thread is
|
|
able to acquire the mutex after the about-to-block
|
|
thread has released it, then a subsequent call to
|
|
pthread_cond_signal() or pthread_cond_broadcast()
|
|
in that thread behaves as if it were issued after
|
|
the about-to-block thread has blocked."
|
|
|
|
Question: Am I right?
|
|
|
|
(I produced the program output above by simply
|
|
adding ?Sleep( 1 )?:
|
|
|
|
================
|
|
waiting threads:
|
|
================
|
|
|
|
{ /** Critical Section
|
|
|
|
inc cond.waiters_count
|
|
|
|
}
|
|
|
|
/*
|
|
/* Atomic only if using Win32 SignalObjectAndWait
|
|
/*
|
|
cond.mtx.release
|
|
|
|
Sleep( 1 ); // Win32
|
|
|
|
/*** ^^-- A THREAD WHICH DID SIGNAL MAY ACQUIRE THE MUTEX,
|
|
/*** GO INTO WAIT ON THE SAME CONDITION AND OVERTAKE
|
|
/*** ORIGINAL WAITER(S) CONSUMING ITS OWN SIGNAL!
|
|
|
|
cond.sem.wait
|
|
|
|
to the source code of pthread-win32 implementation:
|
|
|
|
http://sources.redhat.com/cgi-bin/cvsweb.cgi/pthreads/
|
|
condvar.c?rev=1.36&content-type=text/
|
|
x-cvsweb-markup&cvsroot=pthreads-win32
|
|
|
|
|
|
/*
|
|
* We keep the lock held just long enough to increment the count of
|
|
* waiters by one (above).
|
|
* Note that we can't keep it held across the
|
|
* call to sem_wait since that will deadlock other calls
|
|
* to pthread_cond_signal
|
|
*/
|
|
cleanup_args.mutexPtr = mutex;
|
|
cleanup_args.cv = cv;
|
|
cleanup_args.resultPtr = &result;
|
|
|
|
pthread_cleanup_push (ptw32_cond_wait_cleanup, (void *)
|
|
&cleanup_args);
|
|
|
|
if ((result = pthread_mutex_unlock (mutex)) == 0)
|
|
{((result
|
|
Sleep( 1 ); // @AT
|
|
|
|
/*
|
|
* Wait to be awakened by
|
|
* pthread_cond_signal, or
|
|
* pthread_cond_broadcast, or
|
|
* a timeout
|
|
*
|
|
* Note:
|
|
* ptw32_sem_timedwait is a cancelation point,
|
|
* hence providing the
|
|
* mechanism for making pthread_cond_wait a cancelation
|
|
* point. We use the cleanup mechanism to ensure we
|
|
* re-lock the mutex and decrement the waiters count
|
|
* if we are canceled.
|
|
*/
|
|
if (ptw32_sem_timedwait (&(cv->sema), abstime) == -1) {
|
|
result = errno;
|
|
}
|
|
}
|
|
|
|
pthread_cleanup_pop (1); /* Always cleanup */
|
|
|
|
|
|
BTW, on my system (2 CPUs) I can manage to get
|
|
signals lost even without any source code modification
|
|
if I run the tennis program many times in different
|
|
shell sessions.
|
|
.
|
|
.
|
|
.
|
|
David Schwartz <davids@webmaster.com> wrote:
|
|
>terekhov@my-deja.com wrote:
|
|
>
|
|
>> well, it might be that the program is in fact buggy.
|
|
>> but you did not show me any bug.
|
|
>
|
|
>You're right. I was close but not dead on. I was correct, however,
|
|
>that the code is buggy because it uses 'pthread_cond_signal' even
|
|
>though not any thread waiting on the condition variable can do the
|
|
>job. I was wrong in which thread could be waiting on the cv but
|
|
>unable to do the job.
|
|
|
|
Okay, lets change 'pthread_cond_signal' to 'pthread_cond_broadcast'
|
|
but also add some noise from main() right before declaring the game
|
|
to be over (I need it in order to demonstrate another problem of
|
|
pthread-win32/ACE implementations - broadcast deadlock)...
|
|
.
|
|
.
|
|
.
|
|
It is my understanding of POSIX conditions,
|
|
that on correct implementation added noise
|
|
in form of unnecessary broadcasts from main,
|
|
should not break the tennis program. The
|
|
only 'side effect' of added noise on correct
|
|
implementation would be 'spurious wakeups' of
|
|
players (in fact they are not spurious,
|
|
players just see them as spurious) unblocked,
|
|
not by another player but by main before
|
|
another player had a chance to acquire the
|
|
mutex and change the game state variable:
|
|
.
|
|
.
|
|
.
|
|
|
|
PLAYER-B
|
|
|
|
PLAYER-A
|
|
|
|
---Noise ON...
|
|
|
|
PLAYER-B
|
|
|
|
PLAYER-A
|
|
|
|
.
|
|
.
|
|
.
|
|
|
|
PLAYER-B
|
|
|
|
PLAYER-A
|
|
|
|
----PLAYER-A: SPURIOUS WAKEUP!!!
|
|
|
|
PLAYER-B
|
|
|
|
PLAYER-A
|
|
|
|
---Noise OFF
|
|
|
|
PLAYER-B
|
|
|
|
---Stopping the game...
|
|
|
|
PLAYER-A GONE
|
|
|
|
PLAYER-B GONE
|
|
|
|
GAME OVER
|
|
|
|
H:\SA\UXX\pt\PTHREADS\TESTS>
|
|
|
|
On pthread-win32/ACE implementations the
|
|
program could stall:
|
|
|
|
.
|
|
.
|
|
.
|
|
|
|
PLAYER-A
|
|
|
|
PLAYER-B
|
|
|
|
PLAYER-A
|
|
|
|
PLAYER-B
|
|
|
|
PLAYER-A
|
|
|
|
PLAYER-B
|
|
|
|
PLAYER-A
|
|
|
|
PLAYER-B
|
|
|
|
---Noise ON...
|
|
|
|
PLAYER-A
|
|
|
|
---Noise OFF
|
|
^C
|
|
H:\SA\UXX\pt\PTHREADS\TESTS>
|
|
|
|
|
|
The implementation has problems:
|
|
|
|
================
|
|
waiting threads:
|
|
================
|
|
|
|
{ /** Critical Section
|
|
|
|
inc cond.waiters_count
|
|
|
|
}
|
|
|
|
/*
|
|
/* Atomic only if using Win32 SignalObjectAndWait
|
|
/*
|
|
cond.mtx.release
|
|
cond.sem.wait
|
|
|
|
/*** ^^-- WAITER CAN BE PREEMPTED AFTER BEING UNBLOCKED...
|
|
|
|
{ /** Critical Section
|
|
|
|
dec cond.waiters_count
|
|
|
|
/*** ^^- ...AND BEFORE DECREMENTING THE COUNT (1)
|
|
|
|
last_waiter = ( cond.was_broadcast &&
|
|
cond.waiters_count == 0 )
|
|
|
|
if ( last_waiter )
|
|
|
|
cond.was_broadcast = FALSE
|
|
|
|
endif
|
|
|
|
}
|
|
|
|
if ( last_waiter )
|
|
|
|
/*
|
|
/* Atomic only if using Win32 SignalObjectAndWait
|
|
/*
|
|
cond.auto_reset_event_or_sem.post /* Event for Win32
|
|
cond.mtx.acquire
|
|
|
|
/*** ^^-- ...AND BEFORE CALL TO mtx.acquire (2)
|
|
|
|
/*** ^^-- NESTED BROADCASTS RESULT IN A DEADLOCK
|
|
|
|
|
|
else
|
|
|
|
cond.mtx.acquire
|
|
|
|
/*** ^^-- ...AND BEFORE CALL TO mtx.acquire (3)
|
|
|
|
endif
|
|
|
|
|
|
==================
|
|
broadcast threads:
|
|
==================
|
|
|
|
{ /** Critical Section
|
|
|
|
waiters_count = cond.waiters_count
|
|
|
|
if ( waiters_count != 0 )
|
|
|
|
cond.was_broadcast = TRUE
|
|
|
|
endif
|
|
|
|
}
|
|
|
|
if ( waiters_count != 0 )
|
|
|
|
cond.sem.post waiters_count
|
|
|
|
/*** ^^^^^--- SPURIOUS WAKEUPS DUE TO (1)
|
|
|
|
cond.auto_reset_event_or_sem.wait /* Event for Win32
|
|
|
|
/*** ^^^^^--- DEADLOCK FOR FURTHER BROADCASTS IF THEY
|
|
HAPPEN TO GO INTO WAIT WHILE PREVIOUS
|
|
BROADCAST IS STILL IN PROGRESS/WAITING
|
|
|
|
endif
|
|
|
|
a) cond.waiters_count does not accurately reflect
|
|
number of waiters blocked on semaphore - that could
|
|
result (in the time window when counter is not accurate)
|
|
in spurios wakeups organised by subsequent _signals
|
|
and _broadcasts. From standard compliance point of view
|
|
that is OK but that could be a real problem from
|
|
performance/efficiency point of view.
|
|
|
|
b) If subsequent broadcast happen to go into wait on
|
|
cond.auto_reset_event_or_sem before previous
|
|
broadcast was unblocked from cond.auto_reset_event_or_sem
|
|
by its last waiter, one of two blocked threads will
|
|
remain blocked because last_waiter processing code
|
|
fails to unblock both threads.
|
|
|
|
In the situation with tennisb.c the Player-B was put
|
|
in a deadlock by noise (broadcast) coming from main
|
|
thread. And since Player-B holds the game state
|
|
mutex when it calls broadcast, the whole program
|
|
stalled: Player-A was deadlocked on mutex and
|
|
main thread after finishing with producing the noise
|
|
was deadlocked on mutex too (needed to declare the
|
|
game over)
|
|
|
|
(I produced the program output above by simply
|
|
adding ?Sleep( 1 )?:
|
|
|
|
==================
|
|
broadcast threads:
|
|
==================
|
|
|
|
{ /** Critical Section
|
|
|
|
waiters_count = cond.waiters_count
|
|
|
|
if ( waiters_count != 0 )
|
|
|
|
cond.was_broadcast = TRUE
|
|
|
|
endif
|
|
|
|
}
|
|
|
|
if ( waiters_count != 0 )
|
|
|
|
Sleep( 1 ); //Win32
|
|
|
|
cond.sem.post waiters_count
|
|
|
|
/*** ^^^^^--- SPURIOUS WAKEUPS DUE TO (1)
|
|
|
|
cond.auto_reset_event_or_sem.wait /* Event for Win32
|
|
|
|
/*** ^^^^^--- DEADLOCK FOR FURTHER BROADCASTS IF THEY
|
|
HAPPEN TO GO INTO WAIT WHILE PREVIOUS
|
|
BROADCAST IS STILL IN PROGRESS/WAITING
|
|
|
|
endif
|
|
|
|
to the source code of pthread-win32 implementation:
|
|
|
|
http://sources.redhat.com/cgi-bin/cvsweb.cgi/pthreads/
|
|
condvar.c?rev=1.36&content-type=text/
|
|
x-cvsweb-markup&cvsroot=pthreads-win32
|
|
|
|
if (wereWaiters)
|
|
{(wereWaiters)sroot=pthreads-win32eb.cgi/pthreads/Yem...m
|
|
/*
|
|
* Wake up all waiters
|
|
*/
|
|
|
|
Sleep( 1 ); //@AT
|
|
|
|
#ifdef NEED_SEM
|
|
|
|
result = (ptw32_increase_semaphore( &cv->sema, cv->waiters )
|
|
? 0
|
|
: EINVAL);
|
|
|
|
#else /* NEED_SEM */
|
|
|
|
result = (ReleaseSemaphore( cv->sema, cv->waiters, NULL )
|
|
? 0
|
|
: EINVAL);
|
|
|
|
#endif /* NEED_SEM */
|
|
|
|
}
|
|
|
|
(void) pthread_mutex_unlock(&(cv->waitersLock));
|
|
|
|
if (wereWaiters && result == 0)
|
|
{(wereWaiters
|
|
/*
|
|
* Wait for all the awakened threads to acquire their part of
|
|
* the counting semaphore
|
|
*/
|
|
|
|
if (WaitForSingleObject (cv->waitersDone, INFINITE)
|
|
== WAIT_OBJECT_0)
|
|
{
|
|
result = 0;
|
|
}
|
|
else
|
|
{
|
|
result = EINVAL;
|
|
}
|
|
|
|
}
|
|
|
|
return (result);
|
|
|
|
}
|
|
|
|
BTW, on my system (2 CPUs) I can manage to get
|
|
the program stalled even without any source code
|
|
modification if I run the tennisb program many
|
|
times in different shell sessions.
|
|
|
|
===================
|
|
pthread-win32 patch
|
|
===================
|
|
struct pthread_cond_t_ {
|
|
long nWaitersBlocked; /* Number of threads blocked
|
|
*/
|
|
long nWaitersUnblocked; /* Number of threads unblocked
|
|
*/
|
|
long nWaitersToUnblock; /* Number of threads to unblock
|
|
*/
|
|
sem_t semBlockQueue; /* Queue up threads waiting for the
|
|
*/
|
|
/* condition to become signalled
|
|
*/
|
|
sem_t semBlockLock; /* Semaphore that guards access to
|
|
*/
|
|
/* | waiters blocked count/block queue
|
|
*/
|
|
/* +-> Mandatory Sync.LEVEL-1
|
|
*/
|
|
pthread_mutex_t mtxUnblockLock; /* Mutex that guards access to
|
|
*/
|
|
/* | waiters (to)unblock(ed) counts
|
|
*/
|
|
/* +-> Optional* Sync.LEVEL-2
|
|
*/
|
|
}; /* Opt*) for _timedwait and
|
|
cancellation*/
|
|
|
|
int
|
|
pthread_cond_init (pthread_cond_t * cond, const pthread_condattr_t * attr)
|
|
int result = EAGAIN;
|
|
pthread_cond_t cv = NULL;
|
|
|
|
if (cond == NULL)
|
|
{(cond
|
|
return EINVAL;
|
|
}
|
|
|
|
if ((attr != NULL && *attr != NULL) &&
|
|
((*attr)->pshared == PTHREAD_PROCESS_SHARED))
|
|
{
|
|
/*
|
|
* Creating condition variable that can be shared between
|
|
* processes.
|
|
*/
|
|
result = ENOSYS;
|
|
|
|
goto FAIL0;
|
|
}
|
|
|
|
cv = (pthread_cond_t) calloc (1, sizeof (*cv));
|
|
|
|
if (cv == NULL)
|
|
{(cv
|
|
result = ENOMEM;
|
|
goto FAIL0;
|
|
}
|
|
|
|
cv->nWaitersBlocked = 0;
|
|
cv->nWaitersUnblocked = 0;
|
|
cv->nWaitersToUnblock = 0;
|
|
|
|
if (sem_init (&(cv->semBlockLock), 0, 1) != 0)
|
|
{(sem_init
|
|
goto FAIL0;
|
|
}
|
|
|
|
if (sem_init (&(cv->semBlockQueue), 0, 0) != 0)
|
|
{(sem_init
|
|
goto FAIL1;
|
|
}
|
|
|
|
if (pthread_mutex_init (&(cv->mtxUnblockLock), 0) != 0)
|
|
{(pthread_mutex_init
|
|
goto FAIL2;
|
|
}
|
|
|
|
|
|
result = 0;
|
|
|
|
goto DONE;
|
|
|
|
/*
|
|
* -------------
|
|
* Failed...
|
|
* -------------
|
|
*/
|
|
FAIL2:
|
|
(void) sem_destroy (&(cv->semBlockQueue));
|
|
|
|
FAIL1:
|
|
(void) sem_destroy (&(cv->semBlockLock));
|
|
|
|
FAIL0:
|
|
DONE:
|
|
*cond = cv;
|
|
|
|
return (result);
|
|
|
|
} /* pthread_cond_init */
|
|
|
|
int
|
|
pthread_cond_destroy (pthread_cond_t * cond)
|
|
{
|
|
int result = 0;
|
|
pthread_cond_t cv;
|
|
|
|
/*
|
|
* Assuming any race condition here is harmless.
|
|
*/
|
|
if (cond == NULL
|
|
|| *cond == NULL)
|
|
{
|
|
return EINVAL;
|
|
}
|
|
|
|
if (*cond != (pthread_cond_t) PTW32_OBJECT_AUTO_INIT)
|
|
{(*cond
|
|
cv = *cond;
|
|
|
|
/*
|
|
* Synchronize access to waiters blocked count (LEVEL-1)
|
|
*/
|
|
if (sem_wait(&(cv->semBlockLock)) != 0)
|
|
{(sem_wait(&(cv->semBlockLock))
|
|
return errno;
|
|
}
|
|
|
|
/*
|
|
* Synchronize access to waiters (to)unblock(ed) counts (LEVEL-2)
|
|
*/
|
|
if ((result = pthread_mutex_lock(&(cv->mtxUnblockLock))) != 0)
|
|
{((result
|
|
(void) sem_post(&(cv->semBlockLock));
|
|
return result;
|
|
}
|
|
|
|
/*
|
|
* Check whether cv is still busy (still has waiters blocked)
|
|
*/
|
|
if (cv->nWaitersBlocked - cv->nWaitersUnblocked > 0)
|
|
{(cv->nWaitersBlocked
|
|
(void) sem_post(&(cv->semBlockLock));
|
|
(void) pthread_mutex_unlock(&(cv->mtxUnblockLock));
|
|
return EBUSY;
|
|
}
|
|
|
|
/*
|
|
* Now it is safe to destroy
|
|
*/
|
|
(void) sem_destroy (&(cv->semBlockLock));
|
|
(void) sem_destroy (&(cv->semBlockQueue));
|
|
(void) pthread_mutex_unlock (&(cv->mtxUnblockLock));
|
|
(void) pthread_mutex_destroy (&(cv->mtxUnblockLock));
|
|
|
|
free(cv);
|
|
*cond = NULL;
|
|
}
|
|
else
|
|
{
|
|
/*
|
|
* See notes in ptw32_cond_check_need_init() above also.
|
|
*/
|
|
EnterCriticalSection(&ptw32_cond_test_init_lock);
|
|
|
|
/*
|
|
* Check again.
|
|
*/
|
|
if (*cond == (pthread_cond_t) PTW32_OBJECT_AUTO_INIT)
|
|
{(*cond
|
|
/*
|
|
* This is all we need to do to destroy a statically
|
|
* initialised cond that has not yet been used (initialised).
|
|
* If we get to here, another thread
|
|
* waiting to initialise this cond will get an EINVAL.
|
|
*/
|
|
*cond = NULL;
|
|
}
|
|
else
|
|
{
|
|
/*
|
|
* The cv has been initialised while we were waiting
|
|
* so assume it's in use.
|
|
*/
|
|
result = EBUSY;
|
|
}
|
|
|
|
LeaveCriticalSection(&ptw32_cond_test_init_lock);
|
|
}
|
|
|
|
return (result);
|
|
}
|
|
|
|
/*
|
|
* Arguments for cond_wait_cleanup, since we can only pass a
|
|
* single void * to it.
|
|
*/
|
|
typedef struct {
|
|
pthread_mutex_t * mutexPtr;
|
|
pthread_cond_t cv;
|
|
int * resultPtr;
|
|
} ptw32_cond_wait_cleanup_args_t;
|
|
|
|
static void
|
|
ptw32_cond_wait_cleanup(void * args)
|
|
{
|
|
ptw32_cond_wait_cleanup_args_t * cleanup_args =
|
|
(ptw32_cond_wait_cleanup_args_t *) args;
|
|
pthread_cond_t cv = cleanup_args->cv;
|
|
int * resultPtr = cleanup_args->resultPtr;
|
|
int eLastSignal; /* enum: 1=yes 0=no -1=cancelled/timedout w/o signal(s)
|
|
*/
|
|
int result;
|
|
|
|
/*
|
|
* Whether we got here as a result of signal/broadcast or because of
|
|
* timeout on wait or thread cancellation we indicate that we are no
|
|
* longer waiting. The waiter is responsible for adjusting waiters
|
|
* (to)unblock(ed) counts (protected by unblock lock).
|
|
* Unblock lock/Sync.LEVEL-2 supports _timedwait and cancellation.
|
|
*/
|
|
if ((result = pthread_mutex_lock(&(cv->mtxUnblockLock))) != 0)
|
|
{((result
|
|
*resultPtr = result;
|
|
return;
|
|
}
|
|
|
|
cv->nWaitersUnblocked++;
|
|
|
|
eLastSignal = (cv->nWaitersToUnblock == 0) ?
|
|
-1 : (--cv->nWaitersToUnblock == 0);
|
|
|
|
/*
|
|
* No more LEVEL-2 access to waiters (to)unblock(ed) counts needed
|
|
*/
|
|
if ((result = pthread_mutex_unlock(&(cv->mtxUnblockLock))) != 0)
|
|
{((result
|
|
*resultPtr = result;
|
|
return;
|
|
}
|
|
|
|
/*
|
|
* If last signal...
|
|
*/
|
|
if (eLastSignal == 1)
|
|
{(eLastSignal
|
|
/*
|
|
* ...it means that we have end of 'atomic' signal/broadcast
|
|
*/
|
|
if (sem_post(&(cv->semBlockLock)) != 0)
|
|
{(sem_post(&(cv->semBlockLock))
|
|
*resultPtr = errno;
|
|
return;
|
|
}
|
|
}
|
|
/*
|
|
* If not last signal and not timed out/cancelled wait w/o signal...
|
|
*/
|
|
else if (eLastSignal == 0)
|
|
{
|
|
/*
|
|
* ...it means that next waiter can go through semaphore
|
|
*/
|
|
if (sem_post(&(cv->semBlockQueue)) != 0)
|
|
{(sem_post(&(cv->semBlockQueue))
|
|
*resultPtr = errno;
|
|
return;
|
|
}
|
|
}
|
|
|
|
/*
|
|
* XSH: Upon successful return, the mutex has been locked and is owned
|
|
* by the calling thread
|
|
*/
|
|
if ((result = pthread_mutex_lock(cleanup_args->mutexPtr)) != 0)
|
|
{((result
|
|
*resultPtr = result;
|
|
}
|
|
|
|
} /* ptw32_cond_wait_cleanup */
|
|
|
|
static int
|
|
ptw32_cond_timedwait (pthread_cond_t * cond,
|
|
pthread_mutex_t * mutex,
|
|
const struct timespec *abstime)
|
|
{
|
|
int result = 0;
|
|
pthread_cond_t cv;
|
|
ptw32_cond_wait_cleanup_args_t cleanup_args;
|
|
|
|
if (cond == NULL || *cond == NULL)
|
|
{(cond
|
|
return EINVAL;
|
|
}
|
|
|
|
/*
|
|
* We do a quick check to see if we need to do more work
|
|
* to initialise a static condition variable. We check
|
|
* again inside the guarded section of ptw32_cond_check_need_init()
|
|
* to avoid race conditions.
|
|
*/
|
|
if (*cond == (pthread_cond_t) PTW32_OBJECT_AUTO_INIT)
|
|
{(*cond
|
|
result = ptw32_cond_check_need_init(cond);
|
|
}
|
|
|
|
if (result != 0 && result != EBUSY)
|
|
{(result
|
|
return result;
|
|
}
|
|
|
|
cv = *cond;
|
|
|
|
/*
|
|
* Synchronize access to waiters blocked count (LEVEL-1)
|
|
*/
|
|
if (sem_wait(&(cv->semBlockLock)) != 0)
|
|
{(sem_wait(&(cv->semBlockLock))
|
|
return errno;
|
|
}
|
|
|
|
cv->nWaitersBlocked++;
|
|
|
|
/*
|
|
* Thats it. Counted means waiting, no more access needed
|
|
*/
|
|
if (sem_post(&(cv->semBlockLock)) != 0)
|
|
{(sem_post(&(cv->semBlockLock))
|
|
return errno;
|
|
}
|
|
|
|
/*
|
|
* Setup this waiter cleanup handler
|
|
*/
|
|
cleanup_args.mutexPtr = mutex;
|
|
cleanup_args.cv = cv;
|
|
cleanup_args.resultPtr = &result;
|
|
|
|
pthread_cleanup_push (ptw32_cond_wait_cleanup, (void *) &cleanup_args);
|
|
|
|
/*
|
|
* Now we can release 'mutex' and...
|
|
*/
|
|
if ((result = pthread_mutex_unlock (mutex)) == 0)
|
|
{((result
|
|
|
|
/*
|
|
* ...wait to be awakened by
|
|
* pthread_cond_signal, or
|
|
* pthread_cond_broadcast, or
|
|
* timeout, or
|
|
* thread cancellation
|
|
*
|
|
* Note:
|
|
*
|
|
* ptw32_sem_timedwait is a cancellation point,
|
|
* hence providing the mechanism for making
|
|
* pthread_cond_wait a cancellation point.
|
|
* We use the cleanup mechanism to ensure we
|
|
* re-lock the mutex and adjust (to)unblock(ed) waiters
|
|
* counts if we are cancelled, timed out or signalled.
|
|
*/
|
|
if (ptw32_sem_timedwait (&(cv->semBlockQueue), abstime) != 0)
|
|
{(ptw32_sem_timedwait
|
|
result = errno;
|
|
}
|
|
}
|
|
|
|
/*
|
|
* Always cleanup
|
|
*/
|
|
pthread_cleanup_pop (1);
|
|
|
|
|
|
/*
|
|
* "result" can be modified by the cleanup handler.
|
|
*/
|
|
return (result);
|
|
|
|
} /* ptw32_cond_timedwait */
|
|
|
|
|
|
static int
|
|
ptw32_cond_unblock (pthread_cond_t * cond,
|
|
int unblockAll)
|
|
{
|
|
int result;
|
|
pthread_cond_t cv;
|
|
|
|
if (cond == NULL || *cond == NULL)
|
|
{(cond
|
|
return EINVAL;
|
|
}
|
|
|
|
cv = *cond;
|
|
|
|
/*
|
|
* No-op if the CV is static and hasn't been initialised yet.
|
|
* Assuming that any race condition is harmless.
|
|
*/
|
|
if (cv == (pthread_cond_t) PTW32_OBJECT_AUTO_INIT)
|
|
{(cv
|
|
return 0;
|
|
}
|
|
|
|
/*
|
|
* Synchronize access to waiters blocked count (LEVEL-1)
|
|
*/
|
|
if (sem_wait(&(cv->semBlockLock)) != 0)
|
|
{(sem_wait(&(cv->semBlockLock))
|
|
return errno;
|
|
}
|
|
|
|
/*
|
|
* Synchronize access to waiters (to)unblock(ed) counts (LEVEL-2)
|
|
* This sync.level supports _timedwait and cancellation
|
|
*/
|
|
if ((result = pthread_mutex_lock(&(cv->mtxUnblockLock))) != 0)
|
|
{((result
|
|
return result;
|
|
}
|
|
|
|
/*
|
|
* Adjust waiters blocked and unblocked counts (collect garbage)
|
|
*/
|
|
if (cv->nWaitersUnblocked != 0)
|
|
{(cv->nWaitersUnblocked
|
|
cv->nWaitersBlocked -= cv->nWaitersUnblocked;
|
|
cv->nWaitersUnblocked = 0;
|
|
}
|
|
|
|
/*
|
|
* If (after adjustment) there are still some waiters blocked counted...
|
|
*/
|
|
if ( cv->nWaitersBlocked > 0)
|
|
{(
|
|
/*
|
|
* We will unblock first waiter and leave semBlockLock/LEVEL-1 locked
|
|
* LEVEL-1 access is left disabled until last signal/unblock
|
|
completes
|
|
*/
|
|
cv->nWaitersToUnblock = (unblockAll) ? cv->nWaitersBlocked : 1;
|
|
|
|
/*
|
|
* No more LEVEL-2 access to waiters (to)unblock(ed) counts needed
|
|
* This sync.level supports _timedwait and cancellation
|
|
*/
|
|
if ((result = pthread_mutex_unlock(&(cv->mtxUnblockLock))) != 0)
|
|
{((result
|
|
return result;
|
|
}
|
|
|
|
|
|
/*
|
|
* Now, with LEVEL-2 lock released let first waiter go through
|
|
semaphore
|
|
*/
|
|
if (sem_post(&(cv->semBlockQueue)) != 0)
|
|
{(sem_post(&(cv->semBlockQueue))
|
|
return errno;
|
|
}
|
|
}
|
|
/*
|
|
* No waiter blocked - no more LEVEL-1 access to blocked count needed...
|
|
*/
|
|
else if (sem_post(&(cv->semBlockLock)) != 0)
|
|
{
|
|
return errno;
|
|
}
|
|
/*
|
|
* ...and no more LEVEL-2 access to waiters (to)unblock(ed) counts needed
|
|
too
|
|
* This sync.level supports _timedwait and cancellation
|
|
*/
|
|
else
|
|
{
|
|
result = pthread_mutex_unlock(&(cv->mtxUnblockLock));
|
|
}
|
|
|
|
return(result);
|
|
|
|
} /* ptw32_cond_unblock */
|
|
|
|
int
|
|
pthread_cond_wait (pthread_cond_t * cond,
|
|
pthread_mutex_t * mutex)
|
|
{
|
|
/* The NULL abstime arg means INFINITE waiting. */
|
|
return(ptw32_cond_timedwait(cond, mutex, NULL));
|
|
} /* pthread_cond_wait */
|
|
|
|
|
|
int
|
|
pthread_cond_timedwait (pthread_cond_t * cond,
|
|
pthread_mutex_t * mutex,
|
|
const struct timespec *abstime)
|
|
{
|
|
if (abstime == NULL)
|
|
{(abstime
|
|
return EINVAL;
|
|
}
|
|
|
|
return(ptw32_cond_timedwait(cond, mutex, abstime));
|
|
} /* pthread_cond_timedwait */
|
|
|
|
|
|
int
|
|
pthread_cond_signal (pthread_cond_t * cond)
|
|
{
|
|
/* The '0'(FALSE) unblockAll arg means unblock ONE waiter. */
|
|
return(ptw32_cond_unblock(cond, 0));
|
|
} /* pthread_cond_signal */
|
|
|
|
int
|
|
pthread_cond_broadcast (pthread_cond_t * cond)
|
|
{
|
|
/* The '1'(TRUE) unblockAll arg means unblock ALL waiters. */
|
|
return(ptw32_cond_unblock(cond, 1));
|
|
} /* pthread_cond_broadcast */
|
|
|
|
|
|
|
|
|
|
TEREKHOV@de.ibm.com on 17.01.2001 01:00:57
|
|
|
|
Please respond to TEREKHOV@de.ibm.com
|
|
|
|
To: pthreads-win32@sourceware.cygnus.com
|
|
cc: schmidt@uci.edu
|
|
Subject: win32 conditions: sem+counter+event = broadcast_deadlock +
|
|
spur.wakeup/unfairness/incorrectness ??
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Hi,
|
|
|
|
Problem 1: broadcast_deadlock
|
|
|
|
It seems that current implementation does not provide "atomic"
|
|
broadcasts. That may lead to "nested" broadcasts... and it seems
|
|
that nested case is not handled correctly -> producing a broadcast
|
|
DEADLOCK as a result.
|
|
|
|
Scenario:
|
|
|
|
N (>1) waiting threads W1..N are blocked (in _wait) on condition's
|
|
semaphore.
|
|
|
|
Thread B1 calls pthread_cond_broadcast, which results in "releasing" N
|
|
W threads via incrementing semaphore counter by N (stored in
|
|
cv->waiters) BUT cv->waiters counter does not change!! The caller
|
|
thread B1 remains blocked on cv->waitersDone event (auto-reset!!) BUT
|
|
condition is not protected from starting another broadcast (when called
|
|
on another thread) while still waiting for the "old" broadcast to
|
|
complete on thread B1.
|
|
|
|
M (>=0, <N) W threads are fast enough to go thru their _wait call and
|
|
decrement cv->waiters counter.
|
|
|
|
L (N-M) "late" waiter W threads are a) still blocked/not returned from
|
|
their semaphore wait call or b) were preempted after sem_wait but before
|
|
lock( &cv->waitersLock ) or c) are blocked on cv->waitersLock.
|
|
|
|
cv->waiters is still > 0 (= L).
|
|
|
|
Another thread B2 (or some W thread from M group) calls
|
|
pthread_cond_broadcast and gains access to counter... neither a) nor b)
|
|
prevent thread B2 in pthread_cond_broadcast from gaining access to
|
|
counter and starting another broadcast ( for c) - it depends on
|
|
cv->waitersLock scheduling rules: FIFO=OK, PRTY=PROBLEM,... )
|
|
|
|
That call to pthread_cond_broadcast (on thread B2) will result in
|
|
incrementing semaphore by cv->waiters (=L) which is INCORRECT (all
|
|
W1..N were in fact already released by thread B1) and waiting on
|
|
_auto-reset_ event cv->waitersDone which is DEADLY WRONG (produces a
|
|
deadlock)...
|
|
|
|
All late W1..L threads now have a chance to complete their _wait call.
|
|
Last W_L thread sets an auto-reselt event cv->waitersDone which will
|
|
release either B1 or B2 leaving one of B threads in a deadlock.
|
|
|
|
Problem 2: spur.wakeup/unfairness/incorrectness
|
|
|
|
It seems that:
|
|
|
|
a) because of the same problem with counter which does not reflect the
|
|
actual number of NOT RELEASED waiters, the signal call may increment
|
|
a semaphore counter w/o having a waiter blocked on it. That will result
|
|
in (best case) spurious wake ups - performance degradation due to
|
|
unnecessary context switches and predicate re-checks and (in worth case)
|
|
unfairness/incorrectness problem - see b)
|
|
|
|
b) neither signal nor broadcast prevent other threads - "new waiters"
|
|
(and in the case of signal, the caller thread as well) from going into
|
|
_wait and overtaking "old" waiters (already released but still not returned
|
|
from sem_wait on condition's semaphore). Win semaphore just [API DOC]:
|
|
"Maintains a count between zero and some maximum value, limiting the number
|
|
of threads that are simultaneously accessing a shared resource." Calling
|
|
ReleaseSemaphore does not imply (at least not documented) that on return
|
|
from ReleaseSemaphore all waiters will in fact become released (returned
|
|
from their Wait... call) and/or that new waiters calling Wait... afterwards
|
|
will become less importance. It is NOT documented to be an atomic release
|
|
of
|
|
waiters... And even if it would be there is still a problem with a thread
|
|
being preempted after Wait on semaphore and before Wait on cv->waitersLock
|
|
and scheduling rules for cv->waitersLock itself
|
|
(??WaitForMultipleObjects??)
|
|
That may result in unfairness/incorrectness problem as described
|
|
for SetEvent impl. in "Strategies for Implementing POSIX Condition
|
|
Variables
|
|
on Win32": http://www.cs.wustl.edu/~schmidt/win32-cv-1.html
|
|
|
|
Unfairness -- The semantics of the POSIX pthread_cond_broadcast function is
|
|
to wake up all threads currently blocked in wait calls on the condition
|
|
variable. The awakened threads then compete for the external_mutex. To
|
|
ensure
|
|
fairness, all of these threads should be released from their
|
|
pthread_cond_wait calls and allowed to recheck their condition expressions
|
|
before other threads can successfully complete a wait on the condition
|
|
variable.
|
|
|
|
Unfortunately, the SetEvent implementation above does not guarantee that
|
|
all
|
|
threads sleeping on the condition variable when cond_broadcast is called
|
|
will
|
|
acquire the external_mutex and check their condition expressions. Although
|
|
the Pthreads specification does not mandate this degree of fairness, the
|
|
lack of fairness can cause starvation.
|
|
|
|
To illustrate the unfairness problem, imagine there are 2 threads, C1 and
|
|
C2,
|
|
that are blocked in pthread_cond_wait on condition variable not_empty_ that
|
|
is guarding a thread-safe message queue. Another thread, P1 then places two
|
|
messages onto the queue and calls pthread_cond_broadcast. If C1 returns
|
|
from
|
|
pthread_cond_wait, dequeues and processes the message, and immediately
|
|
waits
|
|
again then it and only it may end up acquiring both messages. Thus, C2 will
|
|
never get a chance to dequeue a message and run.
|
|
|
|
The following illustrates the sequence of events:
|
|
|
|
1. Thread C1 attempts to dequeue and waits on CV non_empty_
|
|
2. Thread C2 attempts to dequeue and waits on CV non_empty_
|
|
3. Thread P1 enqueues 2 messages and broadcasts to CV not_empty_
|
|
4. Thread P1 exits
|
|
5. Thread C1 wakes up from CV not_empty_, dequeues a message and runs
|
|
6. Thread C1 waits again on CV not_empty_, immediately dequeues the 2nd
|
|
message and runs
|
|
7. Thread C1 exits
|
|
8. Thread C2 is the only thread left and blocks forever since
|
|
not_empty_ will never be signaled
|
|
|
|
Depending on the algorithm being implemented, this lack of fairness may
|
|
yield
|
|
concurrent programs that have subtle bugs. Of course, application
|
|
developers
|
|
should not rely on the fairness semantics of pthread_cond_broadcast.
|
|
However,
|
|
there are many cases where fair implementations of condition variables can
|
|
simplify application code.
|
|
|
|
Incorrectness -- A variation on the unfairness problem described above
|
|
occurs
|
|
when a third consumer thread, C3, is allowed to slip through even though it
|
|
was not waiting on condition variable not_empty_ when a broadcast occurred.
|
|
|
|
To illustrate this, we will use the same scenario as above: 2 threads, C1
|
|
and
|
|
C2, are blocked dequeuing messages from the message queue. Another thread,
|
|
P1
|
|
then places two messages onto the queue and calls pthread_cond_broadcast.
|
|
C1
|
|
returns from pthread_cond_wait, dequeues and processes the message. At this
|
|
time, C3 acquires the external_mutex, calls pthread_cond_wait and waits on
|
|
the events in WaitForMultipleObjects. Since C2 has not had a chance to run
|
|
yet, the BROADCAST event is still signaled. C3 then returns from
|
|
WaitForMultipleObjects, and dequeues and processes the message in the
|
|
queue.
|
|
Thus, C2 will never get a chance to dequeue a message and run.
|
|
|
|
The following illustrates the sequence of events:
|
|
|
|
1. Thread C1 attempts to dequeue and waits on CV non_empty_
|
|
2. Thread C2 attempts to dequeue and waits on CV non_empty_
|
|
3. Thread P1 enqueues 2 messages and broadcasts to CV not_empty_
|
|
4. Thread P1 exits
|
|
5. Thread C1 wakes up from CV not_empty_, dequeues a message and runs
|
|
6. Thread C1 exits
|
|
7. Thread C3 waits on CV not_empty_, immediately dequeues the 2nd
|
|
message and runs
|
|
8. Thread C3 exits
|
|
9. Thread C2 is the only thread left and blocks forever since
|
|
not_empty_ will never be signaled
|
|
|
|
In the above case, a thread that was not waiting on the condition variable
|
|
when a broadcast occurred was allowed to proceed. This leads to incorrect
|
|
semantics for a condition variable.
|
|
|
|
|
|
COMMENTS???
|
|
|
|
regards,
|
|
alexander.
|
|
|
|
-----------------------------------------------------------------------------
|
|
|
|
Subject: RE: FYI/comp.programming.threads/Re: pthread_cond_*
|
|
implementation questions
|
|
Date: Wed, 21 Feb 2001 11:54:47 +0100
|
|
From: TEREKHOV@de.ibm.com
|
|
To: lthomas@arbitrade.com
|
|
CC: rpj@ise.canberra.edu.au, Thomas Pfaff <tpfaff@gmx.net>,
|
|
Nanbor Wang <nanbor@cs.wustl.edu>
|
|
|
|
Hi Louis,
|
|
|
|
generation number 8..
|
|
|
|
had some time to revisit timeouts/spurious wakeup problem..
|
|
found some bugs (in 7.b/c/d) and something to improve
|
|
(7a - using IPC semaphores but it should speedup Win32
|
|
version as well).
|
|
|
|
regards,
|
|
alexander.
|
|
|
|
---------- Algorithm 8a / IMPL_SEM,UNBLOCK_STRATEGY == UNBLOCK_ALL ------
|
|
given:
|
|
semBlockLock - bin.semaphore
|
|
semBlockQueue - semaphore
|
|
mtxExternal - mutex or CS
|
|
mtxUnblockLock - mutex or CS
|
|
nWaitersGone - int
|
|
nWaitersBlocked - int
|
|
nWaitersToUnblock - int
|
|
|
|
wait( timeout ) {
|
|
|
|
[auto: register int result ] // error checking omitted
|
|
[auto: register int nSignalsWasLeft ]
|
|
[auto: register int nWaitersWasGone ]
|
|
|
|
sem_wait( semBlockLock );
|
|
nWaitersBlocked++;
|
|
sem_post( semBlockLock );
|
|
|
|
unlock( mtxExternal );
|
|
bTimedOut = sem_wait( semBlockQueue,timeout );
|
|
|
|
lock( mtxUnblockLock );
|
|
if ( 0 != (nSignalsWasLeft = nWaitersToUnblock) ) {
|
|
if ( bTimeout ) { // timeout (or canceled)
|
|
if ( 0 != nWaitersBlocked ) {
|
|
nWaitersBlocked--;
|
|
}
|
|
else {
|
|
nWaitersGone++; // count spurious wakeups
|
|
}
|
|
}
|
|
if ( 0 == --nWaitersToUnblock ) {
|
|
if ( 0 != nWaitersBlocked ) {
|
|
sem_post( semBlockLock ); // open the gate
|
|
nSignalsWasLeft = 0; // do not open the gate below
|
|
again
|
|
}
|
|
else if ( 0 != (nWaitersWasGone = nWaitersGone) ) {
|
|
nWaitersGone = 0;
|
|
}
|
|
}
|
|
}
|
|
else if ( INT_MAX/2 == ++nWaitersGone ) { // timeout/canceled or spurious
|
|
semaphore :-)
|
|
sem_wait( semBlockLock );
|
|
nWaitersBlocked -= nWaitersGone; // something is going on here -
|
|
test of timeouts? :-)
|
|
sem_post( semBlockLock );
|
|
nWaitersGone = 0;
|
|
}
|
|
unlock( mtxUnblockLock );
|
|
|
|
if ( 1 == nSignalsWasLeft ) {
|
|
if ( 0 != nWaitersWasGone ) {
|
|
// sem_adjust( -nWaitersWasGone );
|
|
while ( nWaitersWasGone-- ) {
|
|
sem_wait( semBlockLock ); // better now than spurious
|
|
later
|
|
}
|
|
}
|
|
sem_post( semBlockLock ); // open the gate
|
|
}
|
|
|
|
lock( mtxExternal );
|
|
|
|
return ( bTimedOut ) ? ETIMEOUT : 0;
|
|
}
|
|
|
|
signal(bAll) {
|
|
|
|
[auto: register int result ]
|
|
[auto: register int nSignalsToIssue]
|
|
|
|
lock( mtxUnblockLock );
|
|
|
|
if ( 0 != nWaitersToUnblock ) { // the gate is closed!!!
|
|
if ( 0 == nWaitersBlocked ) { // NO-OP
|
|
return unlock( mtxUnblockLock );
|
|
}
|
|
if (bAll) {
|
|
nWaitersToUnblock += nSignalsToIssue=nWaitersBlocked;
|
|
nWaitersBlocked = 0;
|
|
}
|
|
else {
|
|
nSignalsToIssue = 1;
|
|
nWaitersToUnblock++;
|
|
nWaitersBlocked--;
|
|
}
|
|
}
|
|
else if ( nWaitersBlocked > nWaitersGone ) { // HARMLESS RACE CONDITION!
|
|
sem_wait( semBlockLock ); // close the gate
|
|
if ( 0 != nWaitersGone ) {
|
|
nWaitersBlocked -= nWaitersGone;
|
|
nWaitersGone = 0;
|
|
}
|
|
if (bAll) {
|
|
nSignalsToIssue = nWaitersToUnblock = nWaitersBlocked;
|
|
nWaitersBlocked = 0;
|
|
}
|
|
else {
|
|
nSignalsToIssue = nWaitersToUnblock = 1;
|
|
nWaitersBlocked--;
|
|
}
|
|
}
|
|
else { // NO-OP
|
|
return unlock( mtxUnblockLock );
|
|
}
|
|
|
|
unlock( mtxUnblockLock );
|
|
sem_post( semBlockQueue,nSignalsToIssue );
|
|
return result;
|
|
}
|
|
|
|
---------- Algorithm 8b / IMPL_SEM,UNBLOCK_STRATEGY == UNBLOCK_ONEBYONE
|
|
------
|
|
given:
|
|
semBlockLock - bin.semaphore
|
|
semBlockQueue - bin.semaphore
|
|
mtxExternal - mutex or CS
|
|
mtxUnblockLock - mutex or CS
|
|
nWaitersGone - int
|
|
nWaitersBlocked - int
|
|
nWaitersToUnblock - int
|
|
|
|
wait( timeout ) {
|
|
|
|
[auto: register int result ] // error checking omitted
|
|
[auto: register int nWaitersWasGone ]
|
|
[auto: register int nSignalsWasLeft ]
|
|
|
|
sem_wait( semBlockLock );
|
|
nWaitersBlocked++;
|
|
sem_post( semBlockLock );
|
|
|
|
unlock( mtxExternal );
|
|
bTimedOut = sem_wait( semBlockQueue,timeout );
|
|
|
|
lock( mtxUnblockLock );
|
|
if ( 0 != (nSignalsWasLeft = nWaitersToUnblock) ) {
|
|
if ( bTimeout ) { // timeout (or canceled)
|
|
if ( 0 != nWaitersBlocked ) {
|
|
nWaitersBlocked--;
|
|
nSignalsWasLeft = 0; // do not unblock next waiter
|
|
below (already unblocked)
|
|
}
|
|
else {
|
|
nWaitersGone = 1; // spurious wakeup pending!!
|
|
}
|
|
}
|
|
if ( 0 == --nWaitersToUnblock &&
|
|
if ( 0 != nWaitersBlocked ) {
|
|
sem_post( semBlockLock ); // open the gate
|
|
nSignalsWasLeft = 0; // do not open the gate below
|
|
again
|
|
}
|
|
else if ( 0 != (nWaitersWasGone = nWaitersGone) ) {
|
|
nWaitersGone = 0;
|
|
}
|
|
}
|
|
}
|
|
else if ( INT_MAX/2 == ++nWaitersGone ) { // timeout/canceled or spurious
|
|
semaphore :-)
|
|
sem_wait( semBlockLock );
|
|
nWaitersBlocked -= nWaitersGone; // something is going on here -
|
|
test of timeouts? :-)
|
|
sem_post( semBlockLock );
|
|
nWaitersGone = 0;
|
|
}
|
|
unlock( mtxUnblockLock );
|
|
|
|
if ( 1 == nSignalsWasLeft ) {
|
|
if ( 0 != nWaitersWasGone ) {
|
|
// sem_adjust( -1 );
|
|
sem_wait( semBlockQueue ); // better now than spurious
|
|
later
|
|
}
|
|
sem_post( semBlockLock ); // open the gate
|
|
}
|
|
else if ( 0 != nSignalsWasLeft ) {
|
|
sem_post( semBlockQueue ); // unblock next waiter
|
|
}
|
|
|
|
lock( mtxExternal );
|
|
|
|
return ( bTimedOut ) ? ETIMEOUT : 0;
|
|
}
|
|
|
|
signal(bAll) {
|
|
|
|
[auto: register int result ]
|
|
|
|
lock( mtxUnblockLock );
|
|
|
|
if ( 0 != nWaitersToUnblock ) { // the gate is closed!!!
|
|
if ( 0 == nWaitersBlocked ) { // NO-OP
|
|
return unlock( mtxUnblockLock );
|
|
}
|
|
if (bAll) {
|
|
nWaitersToUnblock += nWaitersBlocked;
|
|
nWaitersBlocked = 0;
|
|
}
|
|
else {
|
|
nWaitersToUnblock++;
|
|
nWaitersBlocked--;
|
|
}
|
|
unlock( mtxUnblockLock );
|
|
}
|
|
else if ( nWaitersBlocked > nWaitersGone ) { // HARMLESS RACE CONDITION!
|
|
sem_wait( semBlockLock ); // close the gate
|
|
if ( 0 != nWaitersGone ) {
|
|
nWaitersBlocked -= nWaitersGone;
|
|
nWaitersGone = 0;
|
|
}
|
|
if (bAll) {
|
|
nWaitersToUnblock = nWaitersBlocked;
|
|
nWaitersBlocked = 0;
|
|
}
|
|
else {
|
|
nWaitersToUnblock = 1;
|
|
nWaitersBlocked--;
|
|
}
|
|
unlock( mtxUnblockLock );
|
|
sem_post( semBlockQueue );
|
|
}
|
|
else { // NO-OP
|
|
unlock( mtxUnblockLock );
|
|
}
|
|
|
|
return result;
|
|
}
|
|
|
|
---------- Algorithm 8c / IMPL_EVENT,UNBLOCK_STRATEGY == UNBLOCK_ONEBYONE
|
|
---------
|
|
given:
|
|
hevBlockLock - auto-reset event
|
|
hevBlockQueue - auto-reset event
|
|
mtxExternal - mutex or CS
|
|
mtxUnblockLock - mutex or CS
|
|
nWaitersGone - int
|
|
nWaitersBlocked - int
|
|
nWaitersToUnblock - int
|
|
|
|
wait( timeout ) {
|
|
|
|
[auto: register int result ] // error checking omitted
|
|
[auto: register int nSignalsWasLeft ]
|
|
[auto: register int nWaitersWasGone ]
|
|
|
|
wait( hevBlockLock,INFINITE );
|
|
nWaitersBlocked++;
|
|
set_event( hevBlockLock );
|
|
|
|
unlock( mtxExternal );
|
|
bTimedOut = wait( hevBlockQueue,timeout );
|
|
|
|
lock( mtxUnblockLock );
|
|
if ( 0 != (SignalsWasLeft = nWaitersToUnblock) ) {
|
|
if ( bTimeout ) { // timeout (or canceled)
|
|
if ( 0 != nWaitersBlocked ) {
|
|
nWaitersBlocked--;
|
|
nSignalsWasLeft = 0; // do not unblock next waiter
|
|
below (already unblocked)
|
|
}
|
|
else {
|
|
nWaitersGone = 1; // spurious wakeup pending!!
|
|
}
|
|
}
|
|
if ( 0 == --nWaitersToUnblock )
|
|
if ( 0 != nWaitersBlocked ) {
|
|
set_event( hevBlockLock ); // open the gate
|
|
nSignalsWasLeft = 0; // do not open the gate below
|
|
again
|
|
}
|
|
else if ( 0 != (nWaitersWasGone = nWaitersGone) ) {
|
|
nWaitersGone = 0;
|
|
}
|
|
}
|
|
}
|
|
else if ( INT_MAX/2 == ++nWaitersGone ) { // timeout/canceled or spurious
|
|
event :-)
|
|
wait( hevBlockLock,INFINITE );
|
|
nWaitersBlocked -= nWaitersGone; // something is going on here -
|
|
test of timeouts? :-)
|
|
set_event( hevBlockLock );
|
|
nWaitersGone = 0;
|
|
}
|
|
unlock( mtxUnblockLock );
|
|
|
|
if ( 1 == nSignalsWasLeft ) {
|
|
if ( 0 != nWaitersWasGone ) {
|
|
reset_event( hevBlockQueue ); // better now than spurious
|
|
later
|
|
}
|
|
set_event( hevBlockLock ); // open the gate
|
|
}
|
|
else if ( 0 != nSignalsWasLeft ) {
|
|
set_event( hevBlockQueue ); // unblock next waiter
|
|
}
|
|
|
|
lock( mtxExternal );
|
|
|
|
return ( bTimedOut ) ? ETIMEOUT : 0;
|
|
}
|
|
|
|
signal(bAll) {
|
|
|
|
[auto: register int result ]
|
|
|
|
lock( mtxUnblockLock );
|
|
|
|
if ( 0 != nWaitersToUnblock ) { // the gate is closed!!!
|
|
if ( 0 == nWaitersBlocked ) { // NO-OP
|
|
return unlock( mtxUnblockLock );
|
|
}
|
|
if (bAll) {
|
|
nWaitersToUnblock += nWaitersBlocked;
|
|
nWaitersBlocked = 0;
|
|
}
|
|
else {
|
|
nWaitersToUnblock++;
|
|
nWaitersBlocked--;
|
|
}
|
|
unlock( mtxUnblockLock );
|
|
}
|
|
else if ( nWaitersBlocked > nWaitersGone ) { // HARMLESS RACE CONDITION!
|
|
wait( hevBlockLock,INFINITE ); // close the gate
|
|
if ( 0 != nWaitersGone ) {
|
|
nWaitersBlocked -= nWaitersGone;
|
|
nWaitersGone = 0;
|
|
}
|
|
if (bAll) {
|
|
nWaitersToUnblock = nWaitersBlocked;
|
|
nWaitersBlocked = 0;
|
|
}
|
|
else {
|
|
nWaitersToUnblock = 1;
|
|
nWaitersBlocked--;
|
|
}
|
|
unlock( mtxUnblockLock );
|
|
set_event( hevBlockQueue );
|
|
}
|
|
else { // NO-OP
|
|
unlock( mtxUnblockLock );
|
|
}
|
|
|
|
return result;
|
|
}
|
|
|
|
---------- Algorithm 8d / IMPL_EVENT,UNBLOCK_STRATEGY == UNBLOCK_ALL ------
|
|
given:
|
|
hevBlockLock - auto-reset event
|
|
hevBlockQueueS - auto-reset event // for signals
|
|
hevBlockQueueB - manual-reset even // for broadcasts
|
|
mtxExternal - mutex or CS
|
|
mtxUnblockLock - mutex or CS
|
|
eBroadcast - int // 0: no broadcast, 1: broadcast, 2:
|
|
broadcast after signal(s)
|
|
nWaitersGone - int
|
|
nWaitersBlocked - int
|
|
nWaitersToUnblock - int
|
|
|
|
wait( timeout ) {
|
|
|
|
[auto: register int result ] // error checking omitted
|
|
[auto: register int eWasBroadcast ]
|
|
[auto: register int nSignalsWasLeft ]
|
|
[auto: register int nWaitersWasGone ]
|
|
|
|
wait( hevBlockLock,INFINITE );
|
|
nWaitersBlocked++;
|
|
set_event( hevBlockLock );
|
|
|
|
unlock( mtxExternal );
|
|
bTimedOut = waitformultiple( hevBlockQueueS,hevBlockQueueB,timeout,ONE );
|
|
|
|
lock( mtxUnblockLock );
|
|
if ( 0 != (SignalsWasLeft = nWaitersToUnblock) ) {
|
|
if ( bTimeout ) { // timeout (or canceled)
|
|
if ( 0 != nWaitersBlocked ) {
|
|
nWaitersBlocked--;
|
|
nSignalsWasLeft = 0; // do not unblock next waiter
|
|
below (already unblocked)
|
|
}
|
|
else if ( 1 != eBroadcast ) {
|
|
nWaitersGone = 1;
|
|
}
|
|
}
|
|
if ( 0 == --nWaitersToUnblock ) {
|
|
if ( 0 != nWaitersBlocked ) {
|
|
set_event( hevBlockLock ); // open the gate
|
|
nSignalsWasLeft = 0; // do not open the gate below
|
|
again
|
|
}
|
|
else {
|
|
if ( 0 != (eWasBroadcast = eBroadcast) ) {
|
|
eBroadcast = 0;
|
|
}
|
|
if ( 0 != (nWaitersWasGone = nWaitersGone ) {
|
|
nWaitersGone = 0;
|
|
}
|
|
}
|
|
}
|
|
else if ( 0 != eBroadcast ) {
|
|
nSignalsWasLeft = 0; // do not unblock next waiter
|
|
below (already unblocked)
|
|
}
|
|
}
|
|
else if ( INT_MAX/2 == ++nWaitersGone ) { // timeout/canceled or spurious
|
|
event :-)
|
|
wait( hevBlockLock,INFINITE );
|
|
nWaitersBlocked -= nWaitersGone; // something is going on here -
|
|
test of timeouts? :-)
|
|
set_event( hevBlockLock );
|
|
nWaitersGone = 0;
|
|
}
|
|
unlock( mtxUnblockLock );
|
|
|
|
if ( 1 == nSignalsWasLeft ) {
|
|
if ( 0 != eWasBroadcast ) {
|
|
reset_event( hevBlockQueueB );
|
|
}
|
|
if ( 0 != nWaitersWasGone ) {
|
|
reset_event( hevBlockQueueS ); // better now than spurious
|
|
later
|
|
}
|
|
set_event( hevBlockLock ); // open the gate
|
|
}
|
|
else if ( 0 != nSignalsWasLeft ) {
|
|
set_event( hevBlockQueueS ); // unblock next waiter
|
|
}
|
|
|
|
lock( mtxExternal );
|
|
|
|
return ( bTimedOut ) ? ETIMEOUT : 0;
|
|
}
|
|
|
|
signal(bAll) {
|
|
|
|
[auto: register int result ]
|
|
[auto: register HANDLE hevBlockQueue ]
|
|
|
|
lock( mtxUnblockLock );
|
|
|
|
if ( 0 != nWaitersToUnblock ) { // the gate is closed!!!
|
|
if ( 0 == nWaitersBlocked ) { // NO-OP
|
|
return unlock( mtxUnblockLock );
|
|
}
|
|
if (bAll) {
|
|
nWaitersToUnblock += nWaitersBlocked;
|
|
nWaitersBlocked = 0;
|
|
eBroadcast = 2;
|
|
hevBlockQueue = hevBlockQueueB;
|
|
}
|
|
else {
|
|
nWaitersToUnblock++;
|
|
nWaitersBlocked--;
|
|
return unlock( mtxUnblockLock );
|
|
}
|
|
}
|
|
else if ( nWaitersBlocked > nWaitersGone ) { // HARMLESS RACE CONDITION!
|
|
wait( hevBlockLock,INFINITE ); // close the gate
|
|
if ( 0 != nWaitersGone ) {
|
|
nWaitersBlocked -= nWaitersGone;
|
|
nWaitersGone = 0;
|
|
}
|
|
if (bAll) {
|
|
nWaitersToUnblock = nWaitersBlocked;
|
|
nWaitersBlocked = 0;
|
|
eBroadcast = 1;
|
|
hevBlockQueue = hevBlockQueueB;
|
|
}
|
|
else {
|
|
nWaitersToUnblock = 1;
|
|
nWaitersBlocked--;
|
|
hevBlockQueue = hevBlockQueueS;
|
|
}
|
|
}
|
|
else { // NO-OP
|
|
return unlock( mtxUnblockLock );
|
|
}
|
|
|
|
unlock( mtxUnblockLock );
|
|
set_event( hevBlockQueue );
|
|
return result;
|
|
}
|
|
---------------------- Forwarded by Alexander Terekhov/Germany/IBM on
|
|
02/21/2001 09:13 AM ---------------------------
|
|
|
|
Alexander Terekhov
|
|
02/20/2001 04:33 PM
|
|
|
|
To: Louis Thomas <lthomas@arbitrade.com>
|
|
cc:
|
|
|
|
From: Alexander Terekhov/Germany/IBM@IBMDE
|
|
Subject: RE: FYI/comp.programming.threads/Re: pthread_cond_* implementatio
|
|
n questions
|
|
Importance: Normal
|
|
|
|
>Sorry, gotta take a break and work on something else for a while.
|
|
>Real work
|
|
>calls, unfortunately. I'll get back to you in two or three days.
|
|
|
|
ok. no problem. here is some more stuff for pauses you might have
|
|
in between :)
|
|
|
|
---------- Algorithm 7d / IMPL_EVENT,UNBLOCK_STRATEGY == UNBLOCK_ALL ------
|
|
given:
|
|
hevBlockLock - auto-reset event
|
|
hevBlockQueueS - auto-reset event // for signals
|
|
hevBlockQueueB - manual-reset even // for broadcasts
|
|
mtxExternal - mutex or CS
|
|
mtxUnblockLock - mutex or CS
|
|
bBroadcast - int
|
|
nWaitersGone - int
|
|
nWaitersBlocked - int
|
|
nWaitersToUnblock - int
|
|
|
|
wait( timeout ) {
|
|
|
|
[auto: register int result ] // error checking omitted
|
|
[auto: register int bWasBroadcast ]
|
|
[auto: register int nSignalsWasLeft ]
|
|
|
|
wait( hevBlockLock,INFINITE );
|
|
nWaitersBlocked++;
|
|
set_event( hevBlockLock );
|
|
|
|
unlock( mtxExternal );
|
|
bTimedOut = waitformultiple( hevBlockQueueS,hevBlockQueueB,timeout,ONE );
|
|
|
|
lock( mtxUnblockLock );
|
|
if ( 0 != (SignalsWasLeft = nWaitersToUnblock) ) {
|
|
if ( bTimeout ) { // timeout (or canceled)
|
|
if ( 0 != nWaitersBlocked ) {
|
|
nWaitersBlocked--;
|
|
nSignalsWasLeft = 0; // do not unblock next waiter
|
|
below (already unblocked)
|
|
}
|
|
else if ( !bBroadcast ) {
|
|
wait( hevBlockQueueS,INFINITE ); // better now than spurious
|
|
later
|
|
}
|
|
}
|
|
if ( 0 == --nWaitersToUnblock ) {
|
|
if ( 0 != nWaitersBlocked ) {
|
|
if ( bBroadcast ) {
|
|
reset_event( hevBlockQueueB );
|
|
bBroadcast = false;
|
|
}
|
|
set_event( hevBlockLock ); // open the gate
|
|
nSignalsWasLeft = 0; // do not open the gate below
|
|
again
|
|
}
|
|
else if ( false != (bWasBroadcast = bBroadcast) ) {
|
|
bBroadcast = false;
|
|
}
|
|
}
|
|
else {
|
|
bWasBroadcast = bBroadcast;
|
|
}
|
|
}
|
|
else if ( INT_MAX/2 == ++nWaitersGone ) { // timeout/canceled or spurious
|
|
event :-)
|
|
wait( hevBlockLock,INFINITE );
|
|
nWaitersBlocked -= nWaitersGone; // something is going on here -
|
|
test of timeouts? :-)
|
|
set_event( hevBlockLock );
|
|
nWaitersGone = 0;
|
|
}
|
|
unlock( mtxUnblockLock );
|
|
|
|
if ( 1 == nSignalsWasLeft ) {
|
|
if ( bWasBroadcast ) {
|
|
reset_event( hevBlockQueueB );
|
|
}
|
|
set_event( hevBlockLock ); // open the gate
|
|
}
|
|
else if ( 0 != nSignalsWasLeft && !bWasBroadcast ) {
|
|
set_event( hevBlockQueueS ); // unblock next waiter
|
|
}
|
|
|
|
lock( mtxExternal );
|
|
|
|
return ( bTimedOut ) ? ETIMEOUT : 0;
|
|
}
|
|
|
|
signal(bAll) {
|
|
|
|
[auto: register int result ]
|
|
[auto: register HANDLE hevBlockQueue ]
|
|
|
|
lock( mtxUnblockLock );
|
|
|
|
if ( 0 != nWaitersToUnblock ) { // the gate is closed!!!
|
|
if ( 0 == nWaitersBlocked ) { // NO-OP
|
|
return unlock( mtxUnblockLock );
|
|
}
|
|
if (bAll) {
|
|
nWaitersToUnblock += nWaitersBlocked;
|
|
nWaitersBlocked = 0;
|
|
bBroadcast = true;
|
|
hevBlockQueue = hevBlockQueueB;
|
|
}
|
|
else {
|
|
nWaitersToUnblock++;
|
|
nWaitersBlocked--;
|
|
return unlock( mtxUnblockLock );
|
|
}
|
|
}
|
|
else if ( nWaitersBlocked > nWaitersGone ) { // HARMLESS RACE CONDITION!
|
|
wait( hevBlockLock,INFINITE ); // close the gate
|
|
if ( 0 != nWaitersGone ) {
|
|
nWaitersBlocked -= nWaitersGone;
|
|
nWaitersGone = 0;
|
|
}
|
|
if (bAll) {
|
|
nWaitersToUnblock = nWaitersBlocked;
|
|
nWaitersBlocked = 0;
|
|
bBroadcast = true;
|
|
hevBlockQueue = hevBlockQueueB;
|
|
}
|
|
else {
|
|
nWaitersToUnblock = 1;
|
|
nWaitersBlocked--;
|
|
hevBlockQueue = hevBlockQueueS;
|
|
}
|
|
}
|
|
else { // NO-OP
|
|
return unlock( mtxUnblockLock );
|
|
}
|
|
|
|
unlock( mtxUnblockLock );
|
|
set_event( hevBlockQueue );
|
|
return result;
|
|
}
|
|
|
|
|
|
----------------------------------------------------------------------------
|
|
|
|
Subject: RE: FYI/comp.programming.threads/Re: pthread_cond_* implementatio
|
|
n questions
|
|
Date: Mon, 26 Feb 2001 22:20:12 -0600
|
|
From: Louis Thomas <lthomas@arbitrade.com>
|
|
To: "'TEREKHOV@de.ibm.com'" <TEREKHOV@de.ibm.com>
|
|
CC: rpj@ise.canberra.edu.au, Thomas Pfaff <tpfaff@gmx.net>,
|
|
Nanbor Wang
|
|
<nanbor@cs.wustl.edu>
|
|
|
|
Sorry all. Busy week.
|
|
|
|
> this insures the fairness
|
|
> which POSIX does not (e.g. two subsequent broadcasts - the gate does
|
|
insure
|
|
> that first wave waiters will start the race for the mutex before waiters
|
|
> from the second wave - Linux pthreads process/unblock both waves
|
|
> concurrently...)
|
|
|
|
I'm not sure how we are any more fair about this than Linux. We certainly
|
|
don't guarantee that the threads released by the first broadcast will get
|
|
the external mutex before the threads of the second wave. In fact, it is
|
|
possible that those threads will never get the external mutex if there is
|
|
enough contention for it.
|
|
|
|
> e.g. i was thinking about implementation with a pool of
|
|
> N semaphores/counters [...]
|
|
|
|
I considered that too. The problem is as you mentioned in a). You really
|
|
need to assign threads to semaphores once you know how you want to wake them
|
|
up, not when they first begin waiting which is the only time you can assign
|
|
them.
|
|
|
|
> well, i am not quite sure that i've fully understood your scenario,
|
|
|
|
Hmm. Well, it think it's an important example, so I'll try again. First, we
|
|
have thread A which we KNOW is waiting on a condition. As soon as it becomes
|
|
unblocked for any reason, we will know because it will set a flag. Since the
|
|
flag is not set, we are 100% confident that thread A is waiting on the
|
|
condition. We have another thread, thread B, which has acquired the mutex
|
|
and is about to wait on the condition. Thus it is pretty clear that at any
|
|
point, either just A is waiting, or A and B are waiting. Now thread C comes
|
|
along. C is about to do a broadcast on the condition. A broadcast is
|
|
guaranteed to unblock all threads currently waiting on a condition, right?
|
|
Again, we said that either just A is waiting, or A and B are both waiting.
|
|
So, when C does its broadcast, depending upon whether B has started waiting
|
|
or not, thread C will unblock A or unblock A and B. Either way, C must
|
|
unblock A, right?
|
|
|
|
Now, you said anything that happens is correct so long as a) "a signal is
|
|
not lost between unlocking the mutex and waiting on the condition" and b) "a
|
|
thread must not steal a signal it sent", correct? Requirement b) is easy to
|
|
satisfy: in this scenario, thread C will never wait on the condition, so it
|
|
won't steal any signals. Requirement a) is not hard either. The only way we
|
|
could fail to meet requirement a) in this scenario is if thread B was
|
|
started waiting but didn't wake up because a signal was lost. This will not
|
|
happen.
|
|
|
|
Now, here is what happens. Assume thread C beats thread B. Thread C looks to
|
|
see how many threads are waiting on the condition. Thread C sees just one
|
|
thread, thread A, waiting. It does a broadcast waking up just one thread
|
|
because just one thread is waiting. Next, before A can become unblocked,
|
|
thread B begins waiting. Now there are two threads waiting, but only one
|
|
will be unblocked. Suppose B wins. B will become unblocked. A will not
|
|
become unblocked, because C only unblocked one thread (sema_post cond, 1).
|
|
So at the end, B finishes and A remains blocked.
|
|
|
|
We have met both of your requirements, so by your rules, this is an
|
|
acceptable outcome. However, I think that the spec says this is an
|
|
unacceptable outcome! We know for certain that A was waiting and that C did
|
|
a broadcast, but A did not become unblocked! Yet, the spec says that a
|
|
broadcast wakes up all waiting threads. This did not happen. Do you agree
|
|
that this shows your rules are not strict enough?
|
|
|
|
> and what about N2? :) this one does allow almost everything.
|
|
|
|
Don't get me started about rule #2. I'll NEVER advocate an algorithm that
|
|
uses rule 2 as an excuse to suck!
|
|
|
|
> but it is done (decrement)under mutex protection - this is not a subject
|
|
> of a race condition.
|
|
|
|
You are correct. My mistake.
|
|
|
|
> i would remove "_bTimedOut=false".. after all, it was a real timeout..
|
|
|
|
I disagree. A thread that can't successfully retract its waiter status can't
|
|
really have timed out. If a thread can't return without executing extra code
|
|
to deal with the fact that someone tried to unblock it, I think it is a poor
|
|
idea to pretend we
|
|
didn't realize someone was trying to signal us. After all, a signal is more
|
|
important than a time out.
|
|
|
|
> when nSignaled != 0, it is possible to update nWaiters (--) and do not
|
|
> touch nGone
|
|
|
|
I realize this, but I was thinking that writing it the other ways saves
|
|
another if statement.
|
|
|
|
> adjust only if nGone != 0 and save one cache memory write - probably much
|
|
slower than 'if'
|
|
|
|
Hmm. You are probably right.
|
|
|
|
> well, in a strange (e.g. timeout test) program you may (theoretically)
|
|
> have an overflow of nWaiters/nGone counters (with waiters repeatedly
|
|
timing
|
|
> out and no signals at all).
|
|
|
|
Also true. Not only that, but you also have the possibility that one could
|
|
overflow the number of waiters as well! However, considering the limit you
|
|
have chosen for nWaitersGone, I suppose it is unlikely that anyone would be
|
|
able to get INT_MAX/2 threads waiting on a single condition. :)
|
|
|
|
Analysis of 8a:
|
|
|
|
It looks correct to me.
|
|
|
|
What are IPC semaphores?
|
|
|
|
In the line where you state, "else if ( nWaitersBlocked > nWaitersGone ) {
|
|
// HARMLESS RACE CONDITION!" there is no race condition for nWaitersGone
|
|
because nWaitersGone is never modified without holding mtxUnblockLock. You
|
|
are correct that there is a harmless race on nWaitersBlocked, which can
|
|
increase and make the condition become true just after we check it. If this
|
|
happens, we interpret it as the wait starting after the signal.
|
|
|
|
I like your optimization of this. You could improve Alg. 6 as follows:
|
|
---------- Algorithm 6b ----------
|
|
signal(bAll) {
|
|
_nSig=0
|
|
lock counters
|
|
// this is safe because nWaiting can only be decremented by a thread that
|
|
// owns counters and nGone can only be changed by a thread that owns
|
|
counters.
|
|
if (nWaiting>nGone) {
|
|
if (0==nSignaled) {
|
|
sema_wait gate // close gate if not already closed
|
|
}
|
|
if (nGone>0) {
|
|
nWaiting-=nGone
|
|
nGone=0
|
|
}
|
|
_nSig=bAll?nWaiting:1
|
|
nSignaled+=_nSig
|
|
nWaiting-=_nSig
|
|
}
|
|
unlock counters
|
|
if (0!=_nSig) {
|
|
sema_post queue, _nSig
|
|
}
|
|
}
|
|
---------- ---------- ----------
|
|
I guess this wouldn't apply to Alg 8a because nWaitersGone changes meanings
|
|
depending upon whether the gate is open or closed.
|
|
|
|
In the loop "while ( nWaitersWasGone-- ) {" you do a sema_wait on
|
|
semBlockLock. Perhaps waiting on semBlockQueue would be a better idea.
|
|
|
|
What have you gained by making the last thread to be signaled do the waits
|
|
for all the timed out threads, besides added complexity? It took me a long
|
|
time to figure out what your objective was with this, to realize you were
|
|
using nWaitersGone to mean two different things, and to verify that you
|
|
hadn't introduced any bug by doing this. Even now I'm not 100% sure.
|
|
|
|
What has all this playing about with nWaitersGone really gained us besides a
|
|
lot of complexity (it is much harder to verify that this solution is
|
|
correct), execution overhead (we now have a lot more if statements to
|
|
evaluate), and space overhead (more space for the extra code, and another
|
|
integer in our data)? We did manage to save a lock/unlock pair in an
|
|
uncommon case (when a time out occurs) at the above mentioned expenses in
|
|
the common cases.
|
|
|
|
As for 8b, c, and d, they look ok though I haven't studied them thoroughly.
|
|
What would you use them for?
|
|
|
|
Later,
|
|
-Louis! :)
|
|
|
|
-----------------------------------------------------------------------------
|
|
|
|
Subject: RE: FYI/comp.programming.threads/Re: pthread_cond_* implementatio
|
|
n questions
|
|
Date: Tue, 27 Feb 2001 15:51:28 +0100
|
|
From: TEREKHOV@de.ibm.com
|
|
To: Louis Thomas <lthomas@arbitrade.com>
|
|
CC: rpj@ise.canberra.edu.au, Thomas Pfaff <tpfaff@gmx.net>,
|
|
Nanbor Wang <nanbor@cs.wustl.edu>
|
|
|
|
Hi Louis,
|
|
|
|
>> that first wave waiters will start the race for the mutex before waiters
|
|
>> from the second wave - Linux pthreads process/unblock both waves
|
|
>> concurrently...)
|
|
>
|
|
>I'm not sure how we are any more fair about this than Linux. We certainly
|
|
>don't guarantee that the threads released by the first broadcast will get
|
|
>the external mutex before the threads of the second wave. In fact, it is
|
|
>possible that those threads will never get the external mutex if there is
|
|
>enough contention for it.
|
|
|
|
correct. but gate is nevertheless more fair than Linux because of the
|
|
barrier it establishes between two races (1st and 2nd wave waiters) for
|
|
the mutex which under 'normal' circumstances (e.g. all threads of equal
|
|
priorities,..) will 'probably' result in fair behaviour with respect to
|
|
mutex ownership.
|
|
|
|
>> well, i am not quite sure that i've fully understood your scenario,
|
|
>
|
|
>Hmm. Well, it think it's an important example, so I'll try again. ...
|
|
|
|
ok. now i seem to understand this example. well, now it seems to me
|
|
that the only meaningful rule is just:
|
|
|
|
a) "a signal is not lost between unlocking the mutex and waiting on the
|
|
condition"
|
|
|
|
and that the rule
|
|
|
|
b) "a thread must not steal a signal it sent"
|
|
|
|
is not needed at all because a thread which violates b) also violates a).
|
|
|
|
i'll try to explain..
|
|
|
|
i think that the most important thing is how POSIX defines waiter's
|
|
visibility:
|
|
|
|
"if another thread is able to acquire the mutex after the about-to-block
|
|
thread
|
|
has released it, then a subsequent call to pthread_cond_signal() or
|
|
pthread_cond_broadcast() in that thread behaves as if it were issued after
|
|
the about-to-block thread has blocked. "
|
|
|
|
my understanding is the following:
|
|
|
|
1) there is no guarantees whatsoever with respect to whether
|
|
signal/broadcast
|
|
will actually unblock any 'waiter' if it is done w/o acquiring the mutex
|
|
first
|
|
(note that a thread may release it before signal/broadcast - it does not
|
|
matter).
|
|
|
|
2) it is guaranteed that waiters become 'visible' - eligible for unblock as
|
|
soon
|
|
as signalling thread acquires the mutex (but not before!!)
|
|
|
|
so..
|
|
|
|
>So, when C does its broadcast, depending upon whether B has started
|
|
waiting
|
|
>or not, thread C will unblock A or unblock A and B. Either way, C must
|
|
>unblock A, right?
|
|
|
|
right. but only if C did acquire the mutex prior to broadcast (it may
|
|
release it before broadcast as well).
|
|
|
|
implementation will violate waiters visibility rule (signal will become
|
|
lost)
|
|
if C will not unblock A.
|
|
|
|
>Now, here is what happens. Assume thread C beats thread B. Thread C looks
|
|
to
|
|
>see how many threads are waiting on the condition. Thread C sees just one
|
|
>thread, thread A, waiting. It does a broadcast waking up just one thread
|
|
>because just one thread is waiting. Next, before A can become unblocked,
|
|
>thread B begins waiting. Now there are two threads waiting, but only one
|
|
>will be unblocked. Suppose B wins. B will become unblocked. A will not
|
|
>become unblocked, because C only unblocked one thread (sema_post cond, 1).
|
|
>So at the end, B finishes and A remains blocked.
|
|
|
|
thread C did acquire the mutex ("Thread C sees just one thread, thread A,
|
|
waiting"). beginning from that moment it is guaranteed that subsequent
|
|
broadcast will unblock A. Otherwise we will have a lost signal with respect
|
|
to A. I do think that it does not matter whether the signal was physically
|
|
(completely) lost or was just stolen by another thread (B) - in both cases
|
|
it was simply lost with respect to A.
|
|
|
|
>..Do you agree that this shows your rules are not strict enough?
|
|
|
|
probably the opposite.. :-) i think that it shows that the only meaningful
|
|
rule is
|
|
|
|
a) "a signal is not lost between unlocking the mutex and waiting on the
|
|
condition"
|
|
|
|
with clarification of waiters visibility as defined by POSIX above.
|
|
|
|
>> i would remove "_bTimedOut=false".. after all, it was a real timeout..
|
|
>
|
|
>I disagree. A thread that can't successfully retract its waiter status
|
|
can't
|
|
>really have timed out. If a thread can't return without executing extra
|
|
code
|
|
>to deal with the fact that someone tried to unblock it, I think it is a
|
|
poor
|
|
>idea to pretend we
|
|
>didn't realize someone was trying to signal us. After all, a signal is
|
|
more
|
|
>important than a time out.
|
|
|
|
a) POSIX does allow timed out thread to consume a signal (cancelled is
|
|
not).
|
|
b) ETIMEDOUT status just says that: "The time specified by abstime to
|
|
pthread_cond_timedwait() has passed."
|
|
c) it seem to me that hiding timeouts would violate "The
|
|
pthread_cond_timedwait()
|
|
function is the same as pthread_cond_wait() except that an error is
|
|
returned if
|
|
the absolute time specified by abstime passes (that is, system time equals
|
|
or
|
|
exceeds abstime) before the condition cond is signaled or broadcasted"
|
|
because
|
|
the abs. time did really pass before cond was signaled (waiter was
|
|
released via semaphore). however, if it really matters, i could imaging
|
|
that we
|
|
can save an abs. time of signal/broadcast and compare it with timeout after
|
|
unblock to find out whether it was a 'real' timeout or not. absent this
|
|
check
|
|
i do think that hiding timeouts would result in technical violation of
|
|
specification.. but i think that this check is not important and we can
|
|
simply
|
|
trust timeout error code provided by wait since we are not trying to make
|
|
'hard' realtime implementation.
|
|
|
|
>What are IPC semaphores?
|
|
|
|
<sys/sem.h>
|
|
int semctl(int, int, int, ...);
|
|
int semget(key_t, int, int);
|
|
int semop(int, struct sembuf *, size_t);
|
|
|
|
they support adjustment of semaphore counter (semvalue)
|
|
in one single call - imaging Win32 ReleaseSemaphore( hsem,-N )
|
|
|
|
>In the line where you state, "else if ( nWaitersBlocked > nWaitersGone ) {
|
|
>// HARMLESS RACE CONDITION!" there is no race condition for nWaitersGone
|
|
>because nWaitersGone is never modified without holding mtxUnblockLock. You
|
|
>are correct that there is a harmless race on nWaitersBlocked, which can
|
|
>increase and make the condition become true just after we check it. If
|
|
this
|
|
>happens, we interpret it as the wait starting after the signal.
|
|
|
|
well, the reason why i've asked on comp.programming.threads whether this
|
|
race
|
|
condition is harmless or not is that in order to be harmless it should not
|
|
violate the waiters visibility rule (see above). Fortunately, we increment
|
|
the counter under protection of external mutex.. so that any (signalling)
|
|
thread which will acquire the mutex next, should see the updated counter
|
|
(in signal) according to POSIX memory visibility rules and mutexes
|
|
(memory barriers). But i am not so sure how it actually works on
|
|
Win32/INTEL
|
|
which does not explicitly define any memory visibility rules :(
|
|
|
|
>I like your optimization of this. You could improve Alg. 6 as follows:
|
|
>---------- Algorithm 6b ----------
|
|
>signal(bAll) {
|
|
> _nSig=0
|
|
> lock counters
|
|
> // this is safe because nWaiting can only be decremented by a thread
|
|
that
|
|
> // owns counters and nGone can only be changed by a thread that owns
|
|
>counters.
|
|
> if (nWaiting>nGone) {
|
|
> if (0==nSignaled) {
|
|
> sema_wait gate // close gate if not already closed
|
|
> }
|
|
> if (nGone>0) {
|
|
> nWaiting-=nGone
|
|
> nGone=0
|
|
> }
|
|
> _nSig=bAll?nWaiting:1
|
|
> nSignaled+=_nSig
|
|
> nWaiting-=_nSig
|
|
> }
|
|
> unlock counters
|
|
> if (0!=_nSig) {
|
|
> sema_post queue, _nSig
|
|
> }
|
|
>}
|
|
>---------- ---------- ----------
|
|
>I guess this wouldn't apply to Alg 8a because nWaitersGone changes
|
|
meanings
|
|
>depending upon whether the gate is open or closed.
|
|
|
|
agree.
|
|
|
|
>In the loop "while ( nWaitersWasGone-- ) {" you do a sema_wait on
|
|
>semBlockLock. Perhaps waiting on semBlockQueue would be a better idea.
|
|
|
|
you are correct. my mistake.
|
|
|
|
>What have you gained by making the last thread to be signaled do the waits
|
|
>for all the timed out threads, besides added complexity? It took me a long
|
|
>time to figure out what your objective was with this, to realize you were
|
|
>using nWaitersGone to mean two different things, and to verify that you
|
|
>hadn't introduced any bug by doing this. Even now I'm not 100% sure.
|
|
>
|
|
>What has all this playing about with nWaitersGone really gained us besides
|
|
a
|
|
>lot of complexity (it is much harder to verify that this solution is
|
|
>correct), execution overhead (we now have a lot more if statements to
|
|
>evaluate), and space overhead (more space for the extra code, and another
|
|
>integer in our data)? We did manage to save a lock/unlock pair in an
|
|
>uncommon case (when a time out occurs) at the above mentioned expenses in
|
|
>the common cases.
|
|
|
|
well, please consider the following:
|
|
|
|
1) with multiple waiters unblocked (but some timed out) the trick with
|
|
counter
|
|
seem to ensure potentially higher level of concurrency by not delaying
|
|
most of unblocked waiters for semaphore cleanup - only the last one
|
|
will be delayed but all others would already contend/acquire/release
|
|
the external mutex - the critical section protected by mtxUnblockLock is
|
|
made smaller (increment + couple of IFs is faster than system/kernel call)
|
|
which i think is good in general. however, you are right, this is done
|
|
at expense of 'normal' waiters..
|
|
|
|
2) some semaphore APIs (e.g. POSIX IPC sems) do allow to adjust the
|
|
semaphore counter in one call => less system/kernel calls.. imagine:
|
|
|
|
if ( 1 == nSignalsWasLeft ) {
|
|
if ( 0 != nWaitersWasGone ) {
|
|
ReleaseSemaphore( semBlockQueue,-nWaitersWasGone ); // better now
|
|
than spurious later
|
|
}
|
|
sem_post( semBlockLock ); // open the gate
|
|
}
|
|
|
|
3) even on win32 a single thread doing multiple cleanup calls (to wait)
|
|
will probably result in faster execution (because of processor caching)
|
|
than multiple threads each doing a single call to wait.
|
|
|
|
>As for 8b, c, and d, they look ok though I haven't studied them
|
|
thoroughly.
|
|
>What would you use them for?
|
|
|
|
8b) for semaphores which do not allow to unblock multiple waiters
|
|
in a single call to post/release (e.g. POSIX realtime semaphores -
|
|
<semaphore.h>)
|
|
|
|
8c/8d) for WinCE prior to 3.0 (WinCE 3.0 does have semaphores)
|
|
|
|
ok. so, which one is the 'final' algorithm(s) which we should use in
|
|
pthreads-win32??
|
|
|
|
regards,
|
|
alexander.
|
|
|
|
----------------------------------------------------------------------------
|
|
|
|
Louis Thomas <lthomas@arbitrade.com> on 02/27/2001 05:20:12 AM
|
|
|
|
Please respond to Louis Thomas <lthomas@arbitrade.com>
|
|
|
|
To: Alexander Terekhov/Germany/IBM@IBMDE
|
|
cc: rpj@ise.canberra.edu.au, Thomas Pfaff <tpfaff@gmx.net>, Nanbor Wang
|
|
<nanbor@cs.wustl.edu>
|
|
Subject: RE: FYI/comp.programming.threads/Re: pthread_cond_* implementatio
|
|
n questions
|
|
|
|
Sorry all. Busy week.
|
|
|
|
> this insures the fairness
|
|
> which POSIX does not (e.g. two subsequent broadcasts - the gate does
|
|
insure
|
|
> that first wave waiters will start the race for the mutex before waiters
|
|
> from the second wave - Linux pthreads process/unblock both waves
|
|
> concurrently...)
|
|
|
|
I'm not sure how we are any more fair about this than Linux. We certainly
|
|
don't guarantee that the threads released by the first broadcast will get
|
|
the external mutex before the threads of the second wave. In fact, it is
|
|
possible that those threads will never get the external mutex if there is
|
|
enough contention for it.
|
|
|
|
> e.g. i was thinking about implementation with a pool of
|
|
> N semaphores/counters [...]
|
|
|
|
I considered that too. The problem is as you mentioned in a). You really
|
|
need to assign threads to semaphores once you know how you want to wake
|
|
them
|
|
up, not when they first begin waiting which is the only time you can assign
|
|
them.
|
|
|
|
> well, i am not quite sure that i've fully understood your scenario,
|
|
|
|
Hmm. Well, it think it's an important example, so I'll try again. First, we
|
|
have thread A which we KNOW is waiting on a condition. As soon as it
|
|
becomes
|
|
unblocked for any reason, we will know because it will set a flag. Since
|
|
the
|
|
flag is not set, we are 100% confident that thread A is waiting on the
|
|
condition. We have another thread, thread B, which has acquired the mutex
|
|
and is about to wait on the condition. Thus it is pretty clear that at any
|
|
point, either just A is waiting, or A and B are waiting. Now thread C comes
|
|
along. C is about to do a broadcast on the condition. A broadcast is
|
|
guaranteed to unblock all threads currently waiting on a condition, right?
|
|
Again, we said that either just A is waiting, or A and B are both waiting.
|
|
So, when C does its broadcast, depending upon whether B has started waiting
|
|
or not, thread C will unblock A or unblock A and B. Either way, C must
|
|
unblock A, right?
|
|
|
|
Now, you said anything that happens is correct so long as a) "a signal is
|
|
not lost between unlocking the mutex and waiting on the condition" and b)
|
|
"a
|
|
thread must not steal a signal it sent", correct? Requirement b) is easy to
|
|
satisfy: in this scenario, thread C will never wait on the condition, so it
|
|
won't steal any signals. Requirement a) is not hard either. The only way
|
|
we
|
|
could fail to meet requirement a) in this scenario is if thread B was
|
|
started waiting but didn't wake up because a signal was lost. This will not
|
|
happen.
|
|
|
|
Now, here is what happens. Assume thread C beats thread B. Thread C looks
|
|
to
|
|
see how many threads are waiting on the condition. Thread C sees just one
|
|
thread, thread A, waiting. It does a broadcast waking up just one thread
|
|
because just one thread is waiting. Next, before A can become unblocked,
|
|
thread B begins waiting. Now there are two threads waiting, but only one
|
|
will be unblocked. Suppose B wins. B will become unblocked. A will not
|
|
become unblocked, because C only unblocked one thread (sema_post cond, 1).
|
|
So at the end, B finishes and A remains blocked.
|
|
|
|
We have met both of your requirements, so by your rules, this is an
|
|
acceptable outcome. However, I think that the spec says this is an
|
|
unacceptable outcome! We know for certain that A was waiting and that C did
|
|
a broadcast, but A did not become unblocked! Yet, the spec says that a
|
|
broadcast wakes up all waiting threads. This did not happen. Do you agree
|
|
that this shows your rules are not strict enough?
|
|
|
|
> and what about N2? :) this one does allow almost everything.
|
|
|
|
Don't get me started about rule #2. I'll NEVER advocate an algorithm that
|
|
uses rule 2 as an excuse to suck!
|
|
|
|
> but it is done (decrement)under mutex protection - this is not a subject
|
|
> of a race condition.
|
|
|
|
You are correct. My mistake.
|
|
|
|
> i would remove "_bTimedOut=false".. after all, it was a real timeout..
|
|
|
|
I disagree. A thread that can't successfully retract its waiter status
|
|
can't
|
|
really have timed out. If a thread can't return without executing extra
|
|
code
|
|
to deal with the fact that someone tried to unblock it, I think it is a
|
|
poor
|
|
idea to pretend we
|
|
didn't realize someone was trying to signal us. After all, a signal is more
|
|
important than a time out.
|
|
|
|
> when nSignaled != 0, it is possible to update nWaiters (--) and do not
|
|
> touch nGone
|
|
|
|
I realize this, but I was thinking that writing it the other ways saves
|
|
another if statement.
|
|
|
|
> adjust only if nGone != 0 and save one cache memory write - probably much
|
|
slower than 'if'
|
|
|
|
Hmm. You are probably right.
|
|
|
|
> well, in a strange (e.g. timeout test) program you may (theoretically)
|
|
> have an overflow of nWaiters/nGone counters (with waiters repeatedly
|
|
timing
|
|
> out and no signals at all).
|
|
|
|
Also true. Not only that, but you also have the possibility that one could
|
|
overflow the number of waiters as well! However, considering the limit you
|
|
have chosen for nWaitersGone, I suppose it is unlikely that anyone would be
|
|
able to get INT_MAX/2 threads waiting on a single condition. :)
|
|
|
|
Analysis of 8a:
|
|
|
|
It looks correct to me.
|
|
|
|
What are IPC semaphores?
|
|
|
|
In the line where you state, "else if ( nWaitersBlocked > nWaitersGone ) {
|
|
// HARMLESS RACE CONDITION!" there is no race condition for nWaitersGone
|
|
because nWaitersGone is never modified without holding mtxUnblockLock. You
|
|
are correct that there is a harmless race on nWaitersBlocked, which can
|
|
increase and make the condition become true just after we check it. If this
|
|
happens, we interpret it as the wait starting after the signal.
|
|
|
|
I like your optimization of this. You could improve Alg. 6 as follows:
|
|
---------- Algorithm 6b ----------
|
|
signal(bAll) {
|
|
_nSig=0
|
|
lock counters
|
|
// this is safe because nWaiting can only be decremented by a thread that
|
|
// owns counters and nGone can only be changed by a thread that owns
|
|
counters.
|
|
if (nWaiting>nGone) {
|
|
if (0==nSignaled) {
|
|
sema_wait gate // close gate if not already closed
|
|
}
|
|
if (nGone>0) {
|
|
nWaiting-=nGone
|
|
nGone=0
|
|
}
|
|
_nSig=bAll?nWaiting:1
|
|
nSignaled+=_nSig
|
|
nWaiting-=_nSig
|
|
}
|
|
unlock counters
|
|
if (0!=_nSig) {
|
|
sema_post queue, _nSig
|
|
}
|
|
}
|
|
---------- ---------- ----------
|
|
I guess this wouldn't apply to Alg 8a because nWaitersGone changes meanings
|
|
depending upon whether the gate is open or closed.
|
|
|
|
In the loop "while ( nWaitersWasGone-- ) {" you do a sema_wait on
|
|
semBlockLock. Perhaps waiting on semBlockQueue would be a better idea.
|
|
|
|
What have you gained by making the last thread to be signaled do the waits
|
|
for all the timed out threads, besides added complexity? It took me a long
|
|
time to figure out what your objective was with this, to realize you were
|
|
using nWaitersGone to mean two different things, and to verify that you
|
|
hadn't introduced any bug by doing this. Even now I'm not 100% sure.
|
|
|
|
What has all this playing about with nWaitersGone really gained us besides
|
|
a
|
|
lot of complexity (it is much harder to verify that this solution is
|
|
correct), execution overhead (we now have a lot more if statements to
|
|
evaluate), and space overhead (more space for the extra code, and another
|
|
integer in our data)? We did manage to save a lock/unlock pair in an
|
|
uncommon case (when a time out occurs) at the above mentioned expenses in
|
|
the common cases.
|
|
|
|
As for 8b, c, and d, they look ok though I haven't studied them thoroughly.
|
|
What would you use them for?
|
|
|
|
Later,
|
|
-Louis! :)
|
|
|