python Multithreading

• Python Threads threading
• • Python Multithreading for I/O intensive
  • Python Multithreaded working process
  • Constructors
  • Thread functions
  • • start()
    • join(timeout=None)
  • Create thread
  • Thread lock
  • 5 Thread lock
  • join function
  • • The first one is ：Python Multithreading defaults （ Non-daemon thread ）
    • The second kind ： The guardian thread
    • The third kind of ： Join in join Method to set synchronization （ Non-daemon thread , And join Don't set the timeout ）
    • A fourth ： Join in join Method to set synchronization （ Non-daemon thread ,join Set timeout ）
    • The fifth ： Join in join Method to set synchronization （* The guardian thread *,join Set timeout ）

Python Threads threading

Python Multithreading for I/O intensive

GIL The full name is Global Interpreter Lock( Global interpreter lock ), For data security ,GIL Ensure that only one thread can get data at the same time . therefore , stay python in , Only one thread can be executed at the same time .

and IO intensive , Multithreading can effectively improve efficiency ( Under single thread IO The operation will go on IO wait for , Cause unnecessary waste of time , And multithreading can be enabled in threads A When waiting for , Automatically switch to thread B, Can not waste CPU Resources for , It can improve the efficiency of program execution ). therefore python Multithreaded pair IO Dense code is more friendly .

and CPU intensive ( All kinds of recycling 、 Calculation and so on ), Because of the calculation work , The timer will soon reach the threshold , then Trigger GIL The release and re competition of （ Switching multiple threads back and forth is of course resource consuming ）, therefore python Multithreaded pair CPU Dense code is not friendly .

Python Multithreaded working process

Python When using multithreading , It's called c The native thread of language .

1. Get public data
2. apply GIL（ Global interpreter lock ）
3. python Interpreter call os Native threads
4. os operation cpu Perform an operation
5. When the thread execution time is up , Whether or not the operation has been performed ,GIL They were asked to release
6. Repeat the above process by other processes
7. Wait for other processes to finish , It will switch to the previous thread （ Continue execution from the context he recorded ）, The whole process is that each thread performs its own operation , When the execution time is up, switch （context switch）.

note ： Some don't understand ？

Constructors

threading.Thread(group=None, target=None, name=None, args=(), kwargs={}, *, daemon=None)

When calling this constructor , Must have a keyword parameter . The parameters are as follows ：

1. group Should be None; For future expansion ThreadGroup Class implementation while retaining .

2. target Is used for run() Method called Callable object ( A function ). The default is None, Indicates that no method needs to be called .

3. name yes Thread name . By default , from “Thread-N” The format forms a unique name , among N It's a small decimal number . Multiple threads can be given the same name .

4. args Is used to call the object function Parameter tuple . The default is ().

5. kwargs Is used to call the object function Keyword parameter Dictionary . The default is {}.

6. daemon If the parameter is not None, Whether the thread is in daemon mode will be explicitly set . If it is None ( The default value is ), Thread will Inherit Of the current thread Daemon mode properties .
   3.3 Version and above only have this attribute .
   Be careful ： Be sure to call start() Set before , Otherwise it will throw RuntimeError .
   The initial value is inherited from the creation thread ; The main thread is not a daemon thread , Therefore, all threads created by the main thread are... By default daemon = False.
   When there are no surviving non daemon threads , Whole Python The program will exit .

7. If subtype ** Overloaded constructor ,** It must ensure that Before doing anything , First initiate Call base class constructor (Thread.init()).

A term is used to explain ： Guardian mode

There is a thread , It's in Background operation Of , Its task is to serve other threads , This kind of thread is called “ Background thread （Daemon Thread）”, Also known as “ The guardian thread ” or “ Sprite threads ”.Python The garbage collection thread of the interpreter is a typical background thread .
Background threads have a feature , If all foreground threads die , Then the background thread will die automatically .

Thread functions

start()

Start thread activity .

It's in One thread Most Can only be called once . It arranges the distribution of objects run() Method is invoked in an independent control process. .

If this method is called in the same thread object More than once , Meeting Throw out RuntimeError .

join(timeout=None)

wait for , Until the end of the thread .

at present Blocking The thread that calls this method , Until it's called join() Thread termination for .

• Whether it's a normal ending
• Or throw an unhandled exception
• Or until a timeout occurs , The timeout option is optional .

When timeout Parameters Exists and is not None when , It should be a to specify the timeout of the operation second Unit Floating point numbers （ Or scores ）.
because join() Always returns None , So you have to be there join() after call is_alive() To determine whether a timeout has occurred , If the thread is still alive , be join() Overtime .

When timeout Parameter does not exist or None , This operation will Block until the thread ends .

One thread Can be join() Many times .

• If Attempting to join the current thread will cause a deadlock , join() Can cause RuntimeError abnormal .
• If you try to join() A thread that has not yet started , The same exception is thrown .

Create thread

import threading # Thread module 
import time
def run(n):
print("task", n)
time.sleep(1)
print('2s')
time.sleep(1)
print('3s')
if __name__ == '__main__':
th = threading.Thread(target=run,name="thread_1" args=("thread 1",), daemon=True) # Create thread 
# Set the subprocess as the daemons ,** Must be in start() Set before 
th.setDaemon(True) # Setting up the guardian thread , In fact, it has been Set up daemon=True
th.start()
# Set the main thread to wait for the child thread to end 
th.join()
print("end")

Thread lock

Because there is random scheduling between threads , And each thread may only execute n After execution , When multiple threads modify the same data at the same time, it may appear Dirty data （ note ：Python Basic data type 、 list 、 Tuples 、 Dictionaries are thread safe , So it will not cause program crash , But it will lead to unknown values in the data , Dirty data ）,

So there's a thread lock , That is, a thread is allowed to perform operations at the same time . Thread locks are used to lock resources , Multiple locks can be defined , Like the code below , When you need to monopolize a resource , Any lock can lock this resource , It's like you can lock the same door with different locks .

Because threads are randomly scheduled , If multiple threads operate on an object at the same time , And the object is not well protected , An unexpected result of the program , So we call it “ Thread unsafe ”.
To prevent this from happening , There's a lock .

5 Thread lock

• Synchronization lock
• Recursive lock
• Conditional lock
• Event lock
• Semaphore lock

I am here 5 Kind of Python Thread lock Detailed in .

join function

Concept supplement ：

• stay Python In multithreaded programming ,join Method is used to synchronize threads
• The guardian thread , It exists to protect others , When it is set as a daemon thread , After the guarded mainline does not exist , The daemon thread does not exist

Following points 5 There are different forms of explanation join Use in multithreaded programming

The first one is ：Python Multithreading defaults （ Non-daemon thread ）

Python Multithreading is the default （ Set thread setDaemon(False)）, After the main thread finishes its task , Dropped out , At this point, the sub thread will continue to perform its own tasks , Until the end of my mission

note ：setDaemon(False) namely This thread is set as a non daemon thread ; After the main program exits , The child thread does not exit automatically .

import threading, time
def doWaiting1():
print('start waiting1: ' + time.strftime('%H:%M:%S') + "\n")
time.sleep(3)
print(" Threads 1 Have been ordered to report ")
print('stop waiting1: ' + time.strftime('%H:%M:%S') + "\n")
def doWaiting2():
print('start waiting2: ' + time.strftime('%H:%M:%S') + "\n")
time.sleep(8)
print(" Threads 2 Have been ordered to report ")
print('stop waiting2: ', time.strftime('%H:%M:%S') + "\n")
tsk = [] # The thread list 
# Create and start threads 1
thread1 = threading.Thread(target = doWaiting1)
thread1.start() # start() function The actual call RUN() function (Python It's called C Thread of language )
tsk.append(thread1)
# Create and start threads 2
thread2 = threading.Thread(target = doWaiting2)
thread2.start()
tsk.append(thread2)
# Timing program 
print('start join: ' + time.strftime('%H:%M:%S') )
print('end join: ' + time.strftime('%H:%M:%S') )

Running results

start waiting1: 20:03:30
start waiting2: 20:03:30
start join: 20:03:30
end join: 20:03:30 # Here the main thread has ended , And the child thread is still working 
Threads 1 Have been ordered to report # Sub thread 1 Keep working 
stop waiting1: 20:03:33
Threads 2 Have been ordered to report
stop waiting2: 20:03:38

Conclusion ：

1. The timer belongs to the main thread , The whole main thread is starting the thread 1 And thread 2 after , Enter the timing module , End of main thread
2. End of main thread , But it doesn't affect threads 1 And thread 2 Operation of , So the following threads 1 And thread 2 Still came to report , At this point, the whole program is completely finished

The second kind ： The guardian thread

Open thread **setDaemon(True),** Set the child thread as the guardian thread , Realize the end of the main program , The subroutine immediately ends all functions

import threading, time
def doWaiting1():
print('start waiting1: ' + time.strftime('%H:%M:%S') + "\n")
time.sleep(3)
print(" Threads 1 Have been ordered to report ")
print('stop waiting1: ' + time.strftime('%H:%M:%S') + "\n")
def doWaiting2():
print('start waiting2: ' + time.strftime('%H:%M:%S') + "\n")
time.sleep(8)
print(" Threads 2 Have been ordered to report ")
print('stop waiting2: ', time.strftime('%H:%M:%S') + "\n")
tsk = []
# Create and start threads 1
thread1 = threading.Thread(target = doWaiting1)
thread1.setDaemon(True)
thread1.start()
tsk.append(thread1)
# Create and start threads 2
thread2 = threading.Thread(target = doWaiting2)
thread2.setDaemon(True)
thread2.start()
tsk.append(thread2)
print('start join: ' + time.strftime('%H:%M:%S') )
print('end join: ' + time.strftime('%H:%M:%S') )

Running results ：

start waiting1: 20:10:04
start waiting2: 20:10:04
start join: 20:10:04
end join: 20:10:04 # After the main thread ends , The child thread ends immediately , In whatever state .

Conclusion ：

1. After the main thread ends , No matter sub thread 1,2 Whether the operation is completed , It's all over and no longer continues to run

The third kind of ： Join in join Method to set synchronization （ Non-daemon thread , And join Don't set the timeout ）

Non-daemon thread , The main program will wait until all subroutines are completed

import threading, time
def doWaiting1():
print('start waiting1: ' + time.strftime('%H:%M:%S') + "\n")
time.sleep(3)
print(" Threads 1 Have been ordered to report ")
print('stop waiting1: ' + time.strftime('%H:%M:%S') + "\n")
def doWaiting2():
print('start waiting2: ' + time.strftime('%H:%M:%S') + "\n")
time.sleep(8)
print(" Threads 2 Have been ordered to report ")
print('stop waiting2: ', time.strftime('%H:%M:%S') + "\n")
tsk = []
# Create and start threads 1
thread1 = threading.Thread(target = doWaiting1)
thread1.start()
tsk.append(thread1)
# Create and start threads 2
thread2 = threading.Thread(target = doWaiting2)
thread2.start()
tsk.append(thread2)
print('start join: ' + time.strftime('%H:%M:%S') )
for t in tsk:
print('%s The thread is here '%t)
t.join() # Threads join() namely ： Two threads run again , You cannot proceed to the next line until all are completed 
print('end join: ' + time.strftime('%H:%M:%S') )

Running results ：

start waiting1: 20:14:35
start waiting2: 20:14:35
start join: 20:14:35
<Thread(Thread-1, started 19648)> The thread is here
Threads 1 Have been ordered to report
stop waiting1: 20:14:38
<Thread(Thread-2, started 24056)> The thread is here
Threads 2 Have been ordered to report
stop waiting2: 20:14:43
end join: 20:14:43 # After the two threads are executed , To run to this 

Conclusion ：

1. Use join function , The main thread will be blocked ( namely ： The main thread , Specify the thread that creates the child thread ), Waiting to be used join Method is completed
2. start join Is in 35 second ,stop waiting1 stay 38 second , just sleep 了 3 second ,stop waiting2 yes 43 second , just sleep 了 8 second , This also shows that , Threads 1 and 2 Is basically running at the same time , However, due to the inconsistent execution time , So the blocking time is also different , Final end join Time is when the last thread runs , The whole program was suspended at 43 second
3. take All threads are put into a list , Loop through all threads in the list join Methods to judge , It is also to ensure that all sub threads can run and complete , The main process just exited

A fourth ： Join in join Method to set synchronization （ Non-daemon thread ,join Set timeout ）

to join Set up timeout The number , It is judged that the child thread has not completed after waiting for more than one time , The main thread no longer waits

note ：join Set after timeout , The judgment is based on Sub thread execution completed | Overtime （ Logical or relationship ）, Which of the two conditions is true first , Just go down .

import threading, time
def doWaiting1():
print('start waiting1: ' + time.strftime('%H:%M:%S') + "\n")
time.sleep(2)
print(" Threads 1 Have been ordered to report ")
print('stop waiting1: ' + time.strftime('%H:%M:%S') + "\n")
def doWaiting2():
print('start waiting2: ' + time.strftime('%H:%M:%S') + "\n")
time.sleep(8)
print(" Threads 2 Have been ordered to report ")
print('stop waiting2: ', time.strftime('%H:%M:%S') + "\n")
tsk = []
# Create and start threads 1
thread1 = threading.Thread(target = doWaiting1)
thread1.start()
tsk.append(thread1)
# Create and start threads 2
thread2 = threading.Thread(target = doWaiting2)
thread2.start()
tsk.append(thread2)
print('start join: ' + time.strftime('%H:%M:%S') )
for t in tsk:
print(" Start :"+time.strftime('%H:%M:%S'))
print('%s The thread is here '%t)
t.join(5)
print(" end :" + time.strftime('%H:%M:%S'))
print('end join: ' + time.strftime('%H:%M:%S') )

Running results ：

start waiting1: 21:14:25
start waiting2: 21:14:25
start join: 21:14:25
Start :21:14:25
<Thread(Thread-1, started 22348)> The thread is here
Threads 1 Have been ordered to report
stop waiting1: 21:14:27
end :21:14:27
Start :21:14:27
<Thread(Thread-2, started 13164)> The thread is here
end :21:14:32
end join: 21:14:32
Threads 2 Have been ordered to report
stop waiting2: 21:14:33

Conclusion ：

1. to join After setting the waiting time , After the waiting time has expired , Main thread terminated , But it does not affect the sub thread to continue running , When all the sub threads have finished running, the whole program terminates
2. The maximum possible waiting time for all threads timeout_total ≤ timeout * Number of threads
3. although timeout The settings are 5s, But threads 1 It only needs 2s, So the loop starts and ends , Consumption only 2s（21:14:27 - 21:14:25）, This cycle goes to the second time , The second wait can still be allocated 5s（21:14:32 - 21:14:27）, So the total waiting time of the two times is 2+5=7s, But threads 2 The time required to run is 8s, and 8s It's from 21:14:25 At the beginning , The end time is 21:14:33, because join The waiting time is over. The main program is over , But it doesn't affect threads 2 Continue operation , So in end join after , Threads 2 Still output the report results （ Because the daemon thread is not enabled ）
4. Individual articles explain join This time is ： The main thread will wait for multiple threads timeout Cumulative sum , This statement is not accurate , from 3 Can be concluded that , Will not necessarily wait “ Number of threads * timeout” So much time , It is ≤“ Number of threads *timeout”,

The fifth ： Join in join Method to set synchronization （ The guardian thread ,join Set timeout ）

Child threads that have timed out and have not finished processing will be terminated directly （ Conform to the characteristics of daemon thread ）

import threading, time
def doWaiting1():
print('start waiting1: ' + time.strftime('%H:%M:%S') + "\n")
time.sleep(2)
print(" Threads 1 Have been ordered to report ")
print('stop waiting1: ' + time.strftime('%H:%M:%S') + "\n")
def doWaiting2():
print('start waiting2: ' + time.strftime('%H:%M:%S') + "\n")
time.sleep(8)
print(" Threads 2 Have been ordered to report ")
print('stop waiting2: ', time.strftime('%H:%M:%S') + "\n")
tsk = []
# Create and start threads 1
thread1 = threading.Thread(target = doWaiting1)
thread1.setDaemon(True) # The guardian thread 
thread1.start()
tsk.append(thread1)
# Create and start threads 2
thread2 = threading.Thread(target = doWaiting2)
thread2.setDaemon(True) # The guardian thread 
thread2.start()
tsk.append(thread2)
print('start join: ' + time.strftime('%H:%M:%S') )
for t in tsk:
print(" Start :"+time.strftime('%H:%M:%S'))
print('%s The thread is here '%t)
t.join(5)
print(" end :" + time.strftime('%H:%M:%S'))
print('end join: ' + time.strftime('%H:%M:%S') )

Running results ：

start waiting1: 21:24:14
start waiting2: 21:24:14
start join: 21:24:14
Start :21:24:14
<Thread(Thread-1, started daemon 9060)> The thread is here
Threads 1 Have been ordered to report
stop waiting1: 21:24:16
end :21:24:16
Start :21:24:16
<Thread(Thread-2, started daemon 13912)> The thread is here
end :21:24:21
end join: 21:24:21

Conclusion ：

1. Compared with the fourth , After timeout, the main thread runs to end join It's over , Sub thread 2 It has been terminated and stopped