Review lessons
First address
Part two address
Part 3 address
Go straight to the point , The third article has mentioned how to receive a reply , The current topic of network programming is about Tcp, The protocol request method itself determines the characteristics , Let's talk about the order of contracting and receiving first .
Tcp The first is a duplex , It means client C Send a message to S End ,S The client can also send messages to the client , In the transmission protocol data type, the transmission in the network card layer is binary file stream . Why is it a stream rather than a package? You can review the previous , You can also choose to memorize .
that C Whether the end is sent to S End message ,S The client must reply , This is not necessarily . Because in the agreement request / The transmission mode also transmits the protocol data type , This type is directly related to the business , It can be roughly divided into response required and no response required . The most common thing that does not require a response is the heartbeat packet . Why is it called a bag instead of a heartbeat , It can only be said to be a common name .
When the client is linked to the service , Every once in a while , Send a fixed message to the server , Just like a person's ECG , Verify that the current client is active . If not active, the server will disconnect the client object （Tcp wave ）, Inactivity here generally refers to how many times you haven't received , Will judge inactive .
Benefits of disconnection , Here you can remember , Every socket Links must have a cost , It's the same to keep the link state , Every socket Doing something will cause memory changes （ Software caching is not considered here , Several levels of hardware cache ）, If you are storing some data into the database , It will exchange the memory data to the database or database cache , The interaction process will also produce memory fragments and database IO expenses . Just active , According to this socket Doing things creates a series of non user overhead , So it's necessary to eliminate inactive requests .
such as 30 Minutes without experience / How many minutes did not move （ Mobile also generates mobile packets , This packet will be synchronized to the active users one by one ）, Then it will enter the hang up state , This is actually similar to fb Mark as inactive (leave = no alive), If you are not active, you only need to synchronize once regularly .
If it is completely offline (close), Business aside , Completely offline will also automatically hang up to gain experience .
The above mainly spread out slowly, and the client sent 3 Time 100 Wait a few hundred bytes ms, But the server only returned a set of packets , This is Tcp Unique sticky bag problem .
But I found that the span is a little big , Examples are more basic , Will come first through a Tcp An example of heartbeat （ Divide into four , Five chapters ） Come on, slowly extend back , Anyway, it will be made clear in the end .
Write an example to do some intuitive reasoning and Design , Examples will also combine some previous knowledge points .
Heartbeat information ：json String # In actual combat, there is a bytes form This butes It's smaller , Just to make the server count, the client is active .
Detailed explanation The server records a management dictionary of client connection handle , The linked client object will be a... On the server side "fd", Whether it is active will be a field "status":"alive",""leave","close" Represent the active , Inactive temporary absence , close .
Development and design on the server side , The support status is a two-way sequence ("alive"<-->"leave"<-->"close")
About status:"alive" State condition It is the default for the client , Not in socket.error That floor was intercepted , So the link is , The link is divided into the correct address , The writing is correct , The firewall allows ( This test doesn't work here )
Server development is required ：
1.socket Server program
2. Regular receive packets
3. Judge 4 The data after bytes is being deserialized json.loads. The server has one more than the client bind Turn on the function of monitoring after , The server supports multiple clients .
4. Match function . The server will match after receiving the client information , There is a cursor （ The first 5 This chapter completes this function ）.
For example, set the cursor to 3, The threshold is 5, How many seconds do you get a message -1, How many seconds without receiving a message +1. The threshold is reached once 5 Count once , achieve 3 Sub threshold , Modify the current client link management dictionary to "leave". The threshold reached is less than 3 So it's still alive.
5. Will use one user The instance object list is used to manage the clients that add simulation links . Every time 10 Seconds plus one , The client is fake , Port number +1.
Need client-side development ：
1.socket client Program
2.Json Contract awarding + data structure
3. Interval every 10 One second .
About status:"leave" state ： Record keeping 30 minute （ Code can write 30 second ） Will go to the next state "close", If the heartbeat is stable many times , The threshold is not triggered 5, Will be back to back "alive".
About arriving status:"close" The client will kick off the server , stay user In the list management of , The server will store the information of the client ip: port .
Write this example first Only "alive" State condition , Without switching conditions . According to past examples , We need to define a data structure of the network layer first , Not this time protobuff.
For protocol transmission Tcp Use high versatility struct, The transmission data structure is Json
Tcp The data structure is packet header 4 Bytes ,4 Bytes are the length of the inner packing body , The whole package length is 4 Bytes （ fixed ）+ baotou 4 The value of bytes （ Dynamic inclusion length ）.
Json C2S {"ip": Pass local Ip,"status"： The status mentioned above uses the default ,"pid": For clients to manage their own } pid This is mainly used to kill your own process .
There is a need to supplement the knowledge , The unfinished part of chapter three , Is that if 4 Byte header length + How the client passes the object is written
# ip_add Address to the server
ip_addr = socket.gethostbyname(socket.gethostname())
# The client manages its own... Locally pid
pid = os.getpid()
# List for state switching , The list is ordered , adopt idx As a state machine index , The default is alive
status = ["alive", "leave", "close"]
idx = 0
data = {'ip': ip_addr, 'status': status[idx], 'pid': pid}
# data The length of , Here you are str Just for the following + Use ,int->str Does not affect the following results
pack_len = str(len(data))
print(pack_len) # Output is 3
# i yes 4 Bytes +pack_len struct.calcsize Yes. fmt Regional
client_len = struct.calcsize("!i" + pack_len + 's')
print(client_len) # Output is 4+3 Pay attention to the notes , The dynamic calculation length comes out , But pay attention to ,struct.pack Packet dynamic is oriented to several objects in the packet header , If only one object is passed in json The output should look like this
This code is about the example of pressure package
buffer = json.dumps(data)
print(len(buffer))
packet_head = struct.pack("!i",len(buffer))
buffer_client = packet_head+buffer.encode("utf-8")
print(buffer_client) # b'\x00\x00\x008{"ip": "192.168.1.105", "status": "alive", "pid": 26464}' It only includes communication with the server , send out Tcp-stuct,Json, hold json The data is sent as a heartbeat packet , Note that the actual heartbeat will indeed be fixed data , But it won't pass in this .
The server just parses the string to determine which client sent the heartbeat . The state switching function will be written in the next article .
import datetime
import socket, sys, os
import time, json,struct
class HeartBeatClient:
interval = 10 # Heartbeat sending interval
def __init__(self, addr: tuple):
try:
self.client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.client.connect(addr)
except socket.error as e:
print("Error creating socket: %s" % e)
sys.exit()
def encode_json(self, conn,data:dict):
""" Contract awarding """
buffer = json.dumps(data)
print(f" The length of packets sent by the current client {len(buffer)}")
packet_head = struct.pack("!i", len(buffer))
conn.send(packet_head + buffer.encode("utf-8"))
def loop_client(self, conn: socket.socket):
"""
Loop client
:param conn:
:return:
"""
status = ["alive", "leave", "close"] # The status list is in order
idx = 0 # Status index
while True:
ip_addr = socket.gethostbyname(socket.gethostname())
pid = os.getpid()
data = {"ip": ip_addr, "status": status[idx], "pid": pid}
buffer = json.dumps(data)
try:
self.encode_json(conn,buffer)
except socket.error:
print("client send failed")
break
else:
now =datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
print({"time":now,"fd":os.getpid(),"status":status[idx]})
time.sleep(self.interval)
self.client_close(conn)
def client_close(self,conn):
if conn:
conn.close()
if __name__ == '__main__':
client = HeartBeatClient(("127.0.0.1", 14000))
client.loop_client(client.client)strengthening strcut Pass examples of use , It is recommended to tap the code by hand .
It mainly corresponds to the heartbeat server function written by the client , Note that the mode is to start the server first , Start the client file .
To simulate fd_port Self increasing 1, It means that many clients connect to , If you are not familiar here Threading Of , Need a little self-study .
import socket
import json
from threading import Thread
import ast
class HeartBeatServer:
def __init__(self, addr: tuple, max: int):
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.sock.bind(addr)
# The biggest support max Number of links
self.sock.listen(max)
self.user = []
def encode_buffer(self, conn: socket.socket, data: str or bytes or dict):
"""
send data
:param conn: The handle that the client links to the server
:param data: data
:return:
"""
if isinstance(data, str):
data = data.encode("utf-8")
elif isinstance(data, dict):
data = str(data).encode("utf-8")
conn.send(str(data).encode("utf-8"))
def match_data(self,recv)->bool:
"""
Match the deserialized data
:param recv:
:return:
"""
if isinstance(recv,dict):
return recv.get("ip") and recv.get("status")
def loop_thread(self, conn: socket.socket, fd_port: int):
"""
The server only judges whether the packet header of the client is legal .
:return:
"""
head_len = 4
self.user.append("192.168.1.105:88888")
idx = 1
while True:
try:
# Keep receiving bags
data = conn.recv(1024 * 4)
if len(data) > head_len:
recv = json.loads(data[4:])
recv = ast.literal_eval(recv)
if not self.match_data(recv):
# pop() The list is thrown out and the lost data is returned , The default is the last
print(f"client fd addr:{self.user.pop()} Connected over!")
break
# To simulate fd_port Self increasing 1, It means that many clients connect to
self.user.append(f"{recv.get('ip')}:{fd_port + idx}")
print(f" Current server management user -->{self.user}")
else:
print(f"client fd addr:{self.user[-1]} Illegal data ")
except socket.error:
print(f"client fd addr:{self.user.pop()} Connected over!")
break
else:
idx += 1
conn.close()
def start(self):
"""
Service startup
:return:
"""
while True:
conn, addr = self.sock.accept()
t = Thread(target=self.loop_thread, args=(conn, addr[1]))
t.start()
if __name__ == '__main__':
server = HeartBeatServer(("127.0.0.1", 14000), 100)
server.start() Why can't I use it here self.sock Because self.sock Is a handle to the server itself
conn Is the handle of the client object received by the server , So this conn Is managed , When the client is disconnected , On the server through conn Wave
If it is self.sock The service was shut down .
The fifth chapter also wrote part of the draft
This article was first published in TesterHome Community , The author is Chen Ziang, a senior game testing and development engineer . use Python Write four articles on network programming , Today's share is the fourth . Link to the original text ： https://testerhome.com/topics/29411
The above is today's sharing , Did you stop learning ~
Want to learn more about dry goods and cutting-edge technology ？
I want to get acquainted with the test industry and the industry elite ？
Welcome to your attention 2022 MTSC The conference （ The 10th China Internet testing and Development Conference ）
Industry pairs MTSC The evaluation of the General Assembly ： to ground 、 Pragmatic 、 There is depth 、 Re share
China Internet test and Development Conference Testing Summit China, By TesterHome Initiated the technical conference of software testing and development industry . There's another alias ：Make Testing Super Cool！
MTSC The main purpose of the conference is to exchange software quality assurance system and testing and R & D technology , Began in 2015 year , Nine sessions have been held successfully . share 1000+ companies ,10000+ Test Engineer 、 Test Manager 、CTO Participation , It has been widely concerned by the whole industry , It's the top conference in China's Internet quality assurance industry .
In order to ensure the quality of the topic , Every year, topics will be collected half a year in advance , After several months of review , Finalize the topics to be shared at the conference .MTSC 2022 The topics are still being solicited , We sincerely invite all senior test technology experts 、 Quality management managers and test rookies recommend themselves / Recommended topics ！
Just click https://www.wjx.top/vj/wZwCju... Enter the delivery entrance , Fill in and submit according to the format , Welcome to introduce yourself / Recommended topics .
In order to facilitate the review team to have more time for review and communication with lecturers , Present better topics for the majority of participants , Topic submission （ But there is no PPT, There is an outline and other topic information introduction ） The delivery deadline is advanced to ：2022 year 3 month 30 Japan
Overall process ： Case submission > Preliminary examination and approval > PPT Submit > Confirm the topic > Conference speech
Case submission >> 3 month 30 Japan
Preliminary examination and approval >> 4 month 15 Japan
ppt Submit >> 5 month 15 Japan
Topic confirmation >> 5 month 30 Japan
Conference speech >> 7 month 22~23 Japan
MTSC 2022 Early bird tickets have been quietly on sale , Click to learn more .
