python because GIL（ Global lock ） The existence of , Can't take advantage of multi-core , Performance has been criticized .
Solution 1 ：
use multiprocessing replace Thread
multiprocessing The emergence of the library is largely to make up for thread Cuz GIL And inefficiency . It makes a complete copy of thread The interface provided is easy to migrate . The only difference is that it uses multiple processes instead of multiple threads . Each process has its own independent GIL, So there will be no inter process GIL Scramble for .
multiprocessing Not everything . Its introduction will increase the difficulty of data communication and synchronization between threads in program implementation . Take the counter for example , If we want multiple threads to accumulate the same variable , about thread Come on , Make a statement global Variable can . and multiprocessing Because processes cannot access each other's data , Only one... Can be declared in the main thread Queue,put Again get Or use share memory Methods . This extra cost of implementation makes it very painful to code multithreaded programs , Become more painful .
Solution 2 ：
Python Multithreading in multi-core CPU On , Only for IO Intensive computing has a positive effect ; And when there is at least one CPU Dense threads exist , So many threads are efficient because GIL And a big drop .
Official documents ：
asyncio Official documents .
git Warehouse ：
asyncio git Warehouse .
asyncio It's for writing Concurrent Code library , Use async/await grammar .
asyncio Used to provide high performance for multiple Python The foundation of asynchronous framework , Including Internet and web services , Database connection Library , Distributed task queues and so on .
asyncio It's often about building IO Intensive and high-level structured The best choice of network code .
asyncio Provide a set of High level API be used for :
Concurrently function Python coroutines And realize complete control over its execution process ;
perform The Internet IO and IPC;
control Subprocesses ;
adopt queue Implement distributed tasks ;
Sync Concurrent code ;
Besides , Some more Lower level API To support the Developers of libraries and frameworks Realization :
Create and manage The event loop , To provide asynchronous API be used for Networking , function Subprocesses , Handle OS The signal wait ;
Use transports Achieve efficient protocols ;
adopt async/await grammar The bridge Callback based libraries and code .
coroutines
adopt async/await Syntax to declare , It's writing asyncio Recommended way to apply . for example , The following code snippet （ need Python 3.7+） prints “hello”, wait for 5 second , Re print “world”:
import asyncio
import time
async def main():
print('hello')
print(f"started at {time.strftime('%X')}")
await asyncio.sleep(5)
print(f"end at {time.strftime('%X')}")
print('world')
asyncio.run(main())
asynchronous http service ：
import time
import aiohttp,asyncio
async def request_demo():
async with aiohttp.ClientSession() as session:
async with session.get('http://www.baidu.com/',verify_ssl=False) as response:
print('status:',response.status)
print('content-type',response.headers['content-type'])
html=await response.text()
print(f'body:{html[:15]}')
# Create an event loop
loop=asyncio.get_event_loop()
tasks=[request_demo(),]
loop.run_until_complete(asyncio.wait(tasks))
