Introduction to network programming with Python #0001 UDP (2020 Tutorial)
Table of Contents
1. Introduction to network communication
2. IP address
3. Linux command (ping, ifconfig)
4. Port
5. Socket
6. UDP application
7. Python 3 encoding and decoding
8. Connection between UPD and Port
9. Practice, a UDP chatter
1. Introduction to network communication
In this tutorial, we are gonna introduce our first topic on network communication UDP. In the next tutorial, we are gonna introduce the second topic TCP which is more complex than UDP. I would like to keep the this tutorial as simple as possible by just using plain words. Our main purpose is to create applications that could communicate with each other through network.
1.1 What is network?
As you can see from the picture, a man holding a walkie-talkie talk to another person hundreds of meters away. Words are exchanged between two people through radio waves which formed a network. So in plain word, as long as there are at least two devices that could send information back and forth, they form a network.
A network doesn’t have to be wired by physical wires or fibre, such as wifi, bluetooth etc.
1.2 Why do we need a network?
As you can see from the picture above. Back in 80s and early 90s, when I was a kid, I liked to play video games on my Nintendo. Sometimes I would like to invite my best friends to play with me, because it was really fun to enjoy the games with them. Back then, there was no network that would connect my device with my friend’s, so we just play alone all by myself all the time. So here is the network that came to rescue.
So, the main purpose of network is to connect all the devices together so as to make each of them communicate with each other.
As for network programming, it is to create a program on a computer that could exchange data with another program that talks the same protocol in another computer in a local network or on the internet.
As you can see from the picture, we have two computers inside my house which forms a local network. The two computers could communicate with each other by locating each other with their specific IP addresses (topic we later introduce). If you want to talk with another computer with a IP address (55.49.110.89) in New York, you local network has to be connected to the Internet which is just a larger network consisting of millions of small local networks.
2. IP Address
2.1 What is IP?
In real world, if you want to send a package from LA to a friend in New York, you must fill the delivery form with the correct address. If you fill in the wrong address, you friend will miss the package and be mad of you. Similarly if you want to send a message through network to your friend’s messenger app, you need to know the address of your friend’s computer where his messenger app resides. So this address is called the IP address in the network.
2.2 What does an IP look like?
IPv4 and IPv6
There are two versions of IP addressing scheme, IPv4 and IPv6. For this tutorial we only talk about IPv4. IPv6 is new and has much more addresses (theoretically 2128) than IPv4, while IPv4 has only over 400 million addresses.
As you can see, an IPv4 address has only 4 bytes, each byte is separated by dot ‘.’. So, each slot could be integers from 0 to 255.
TCP/IP defines five classes of IP addresses (ABCDE). Each class has a valid range of IP addresses. IP addresses from class A, B and C can be used to ID hosts. See the table below.
Class | First Octet Value | Usage |
A | 0-127 | 127.25.211.36 |
B | 128-191 | 168.1.25.65 |
C | 192-223 | 192.168.1.100 |
D | 224-239 | multicast |
E | 240-256 | experimental purpose |
Class A address use the first octet to ID a network, last three octets to ID hosts.
Example:
126.15.65.96: This is a class A address, ‘126’ is used to ID the network, while ‘15.65.96’ to ID a host. So a class A address could ID 256*256*256 hosts.
Class B address use the two slots to ID the network, last two slots to ID hosts.
Example:
168.32.123.100: This is a class B address, ‘168.32’ is used to ID the network, while ‘123.100’ to ID a host. So a class B address could ID 256*256 hosts
class C address use the the first three slots to ID a network, the last slot to ID the hosts.
Example:
192.168.123.100: This is a class C address, ‘192.168.123’ is used to ID the network, while ‘100’ to ID the host.
If we have two class rooms, each room has a separate local network. Both networks connect to the larger school local network. How could we differentiate the two local networks by just looking at their addresses.
As you can see, classroom A has a bunch of computers and each computer’s IP address starts with 192.168.33.XXX. While IP addresses in classroom B starts with 192.168.34.XXX. So we can tell that computers with address 192.168.33.XXX are from classroom A and those with address 192.168.34.XXX are from classroom B. And those addresses are of class C type.
In classroom A’s local network, IP addresses starts with the same 192.168.33.XXX, so we could mark 256 computers with unique addresses in theory, that is 192.168.33.0-192.168.33.255. We use the first three slots to ID the same local network leaving the last slot to ID each host in the local network.
If our network has only a few devices (less than 256), the strategy to ID the network by using three slots of IPv4 address is just fine. However, if your firm has over 10000 computers or hosts, it is obvious the strategy above is not applicable. So we use another strategy by using the first two slot of the IPv4 address to ID the network and the last two slot to ID the hosts in your firm. Then you could have 256*256 = 65536 unique addresses to dispose.
Similarly, if your network is even larger, like those in Google, Facebook, etc., you could just use one slot to ID the network and all three other slots to ID the hosts within your network. There are 256*256*256=16777216 unique addresses.
2.3 How does a message travel through a network
Just like the picture, computer A has an IP address 192.168.1.100 and wants to send a message ‘dinner tonight?’ to computer C with an IP address 192.168.1.103.
First, computer A puts the message and the computer C’s IP address into a package and sends it to the router.
Second, the router checks the IP address that comes with the message and find the route to the specific IP address and directs the package to the route toward computer C.
3. Linux command (ping, ifconfig)
How to check your own IP address? Just use the following commands for linux and windows.
Linux:
ifconfig
Windows:
ipconfig
4. Port
In order to send the message ‘dinner tonight?’ to the computer C, we need other information alongside the IP address.
As in the picture, our computer A has a lot of messaging apps, e.g. WhatsApp, Skype, Telegram, Messenger. We want to send our message through A’s Skype to B’s Skype at 192.168.1.103. How could the message find the right software? Then port comes to the rescue.
When you launch an web application, your app will bind a port number (e.g 7878) to itself. And this is the port with that our OS will deliver the message to the right app.
If the Skype is bound to a port number 4455 in computer A and 7878 in computer B, so our message will be packaged with the following information:
l dest_ip: 192.168.1.103
l The IP address of the destination computer
l src_ip: 192.18.1.100
l The IP address of the source computer
l dest_port: 7878
l The port number of the destination computer
l src_port: 4455
l The port number of the source computer
l content: ‘diner tonight?’
First, package goes to the router and the router decides which way should this message go by matching the dest_ip against others in the network.
Second, package is sent to the right computer with the same dest_ip.
Third, OS sends the message to the right Skype app by matching dest_port against ports bound to all the apps launched or processes. And destination Skype receive the message. Then, it uses the src_ip and src_port to send the response back to the source computer A.
4.1 Well Known Ports
On Linux, we could have 0 - 65535 ports. For well-known ports, it is 0 - 1023. Those ports are reserved for specific services:
port service
80 HTTP Service
21 FTP Service
443 HTTPS Service
Those ports cannot be used without root permission.
1.1 Dynamic Ports
Ports between 1024 and 65535 are dynamic ports. Launched apps or processes can pick a dynamic port number and release the port number when the process is closed.
5. Socket
In previous topics, we know that IP address could ID the computer in a network. Protocol and port could ID the process (namely launched app and its associated resources) running in the computer. Then socket comes into play.
A socket is just a software that we could use in Python to send and receive data from a another computer in a network. By using socket, we don’t have to worry about all the lower level details when transmitting data.
Its usage is simple:
lopen a socket
lread from and write to the socket
lclose the socket
import socket # 1. open a socket by creating a socket object s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # 2. read from and write to socket # code goes here # 3. close the socket s.close()
6. UDP application
Now, we could use the socket to create a UDP application that could talk with another UDP app on a different computer in the same local network.
# a basic udp client import socket # create a udp socket udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # can only send bytes-like object, not str type data_to_send = b'dinner tonight?' dest_ip = '192.168.123.64' dest_port = 8080 # send data to the destination udp_socket.sendto(data_to_send, (dest_ip, dest_port)) # close the socket udp_socket.close()
Most of the codes are self-explanatory. When we create a socket object, we pass socket.AF_INET as the first argument which indicates that we will use IPv4 and pass socket.SOCK_DGRAM as the second argument which means the socket will use UPD protocol to talk with another computer.
We use socket method udp_socket.sendto() to send data. The first argument is a bytes-like object that represents our message; and second argument is a tuple consisting both the IP address and the port.
In order to test our application, we use a handy Net Assistant app installed on another machine with IP address ‘192.168.123.64’. You can download this app from here.
In order to receive to the message, we must set its Protocol to UDP and port to 8080, and then click open. When run the python file, the Net Assistant will receive the message ‘dinner tonight?’ and have it displayed on the right white space.
Wow, it works and it is simple.
Next, we need to improve this application by letting user input from key board.
# a basic udp client with user input import socket # create a udp socket udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # can only send bytes-like object, not str type # data_to_send = b'dinner tonight?' data_to_send = input('Enter your message: ') dest_ip = '192.168.123.64' dest_port = 8080 # send data to the destination # we need to convert string to bytes-like object by using str method encode() udp_socket.sendto(data_to_send.encode('utf-8'), (dest_ip, dest_port)) # close the socket udp_socket.close()
When you run the script and enter ‘hello’, our network assistant will receive the data.
And finally, we want to send message over and over again. Here comes the while loop.
# a basic udp client with user input and while True loop import socket # create a udp socket udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # can only send bytes-like object, not str type # data_to_send = b'dinner tonight?' dest_ip = '192.168.123.64' dest_port = 8080 while True: data_to_send = input('Enter your message: ') # if user enter exit, quit the application if data_to_send == 'exit': break # send data to the destination # we need to convert string to bytes-like object by using str method encode() udp_socket.sendto(data_to_send.encode('utf-8'), (dest_ip, dest_port)) # close the socket udp_socket.close()
Then we should be able to send data over and over again.
7. Python 3 encoding and decoding
As you could see from the program, our socket object could only send and receive bytes instead of str objects. Fortunately, Python provides us str.encode() method and bytes.decode() method to do the trick.
Explanation from the office document:
str.encode(encoding='utf-8', errors='strict')
Return an encoded version of the string as a bytes object. Default encoding is 'utf-8'. errors may be given to set a different error handling scheme. The default for errors is 'strict', meaning that encoding errors raise a UnicodeError. Other possible values are 'ignore', 'replace', 'xmlcharrefreplace', 'backslashreplace' and any other name registered via codecs.register_error(), see section Error Handlers. For a list of possible encodings, see section Standard Encodings.
bytes.decode(encoding='utf-8', errors='strict')
Return a string decoded from the given bytes. Default encoding is 'utf-8'. errors may be given to set a different error handling scheme. The default for errors is 'strict', meaning that encoding errors raise a UnicodeError. Other possible values are 'ignore', 'replace' and any other name registered via codecs.register_error(), see section Error Handlers. For a list of possible encodings, see section Standard Encodings.
For most cases, ‘utf-8’ encoding is enough. If you have encoding issues, please check the default encoding of your system.
8. Connection between UPD and Port
From previous section, we could send data to a UDP client; now we need to create a UDP app to receive the data sent by another UDP app.
In order to receive data, we need our socket object to bind a local IP and a port.
# a basic udp client to recieve data import socket # create a udp socket udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # bind a local IP and port local_ip = '' # usually leave it as empty, if you have only one local IP local_port = 7878 udp_socket.bind((local_ip, local_port)) # receive data from a remote UDP app data_recv = udp_socket.recvfrom(1024) # receive 1024 bytes # print the data_recv print(data_recv) # close the socket udp_socket.close()
We use the bind() method which takes a tuple consisting of the local ip and local port as arguments to bind our socket with the local machine. Then we use recvfrom() method to receive the data. It takes the number of bytes to be received as argument. Then we print the data.
After we run the script, the program will halt waiting for the data.
Next, we open our Network Assistant app on the same local machine or another machine in the same local network.
Open a UDP on a port other than 7878. Then change the IP address and port of the right-bottom section to the address and port (7878) of your UDP app. Type some text into the text input area of the right-bottom section. And click ‘Send’ button. Voila! You will see the data received from your python shell.
As you can see, you receive a tuple:
(b'hello', ('192.168.123.64', 8080))
Next, we can fetch each piece of information by using indexes.
>>> data_recv[0]
b’hello’
>>> data_recv[1]
('192.168.123.64', 8080)
Finally, let’s add a while loop to constantly receive data.
# a basic udp client to recieve import socket # create a udp socket udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # bind a local IP and port local_ip = '' # usually leave it as empty, if you have only one local IP local_port = 7878 udp_socket.bind((local_ip, local_port)) while True: # receive data from a remote UDP app data_recv = udp_socket.recvfrom(1024) # receive 1024 bytes msg = data_recv[0].decode('utf-8') # data need to be decoded ip_port = str(data_recv[1]) # print the data_recv print(ip_port, ': ', msg) # close the socket udp_socket.close()
As you can see from the code, the message received is a bytes-like object, we need to decode it using ‘utf-8’.
Congratulations, your app should work fine when you run it. If no error pops out, you could use your Network Assistant app to constantly send data to this app.
Further notice on port binding
In the previous topic, we wrote an app that constantly send message to our Network Assistant:
# a basic udp client with user input and while True loop import socket # create a udp socket udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # can only send bytes-like object, not str type # data_to_send = b'dinner tonight?' dest_ip = '192.168.123.64' dest_port = 8080 while True: data_to_send = input('Enter your message: ') # if user enter exit, quit the application if data_to_send == 'exit': break # send data to the destination # we need to convert string to bytes-like object by using str method encode() udp_socket.sendto(data_to_send.encode('utf-8'), (dest_ip, dest_port)) # close the socket udp_socket.close()
In this case, we didn’t bind any IP or local port to our socket object. But as we can see from the screenshot, our app is automatically bind to the port 54510. If you want to bind a specific port such as 7878, you could use bind() method to achieve this end.
# a basic udp client with user input and while True loop # socket bind to port 7878 import socket # create a udp socket udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # bind socket to port 7878 local_ip = '' # usually leave it as an empty string if there is one ip address local_port = 7878 udp_socket.bind((local_ip, local_port)) # can only send bytes-like object, not str type # data_to_send = b'dinner tonight?' dest_ip = '192.168.123.64' dest_port = 8080 while True: data_to_send = input('Enter your message: ') # if user enter exit, quit the application if data_to_send == 'exit': break # send data to the destination # we need to convert string to bytes-like object by using str method encode() udp_socket.sendto(data_to_send.encode('utf-8'), (dest_ip, dest_port)) # close the socket udp_socket.close()
In this program, we bind the local address to the socket object.
local_ip = '' # usually leave it as an empty string if there is one ip address
local_port = 7878
udp_socket.bind((local_ip, local_port))
After running this, you can see our port 7878 is used instead of a randomly picked port number.
9. A UDP chatter
Before we write a full-fledged UDP chatter, let’s write another simple UDP app which could send and receive data with one socket.
# udp basic chatter import socket # create a udp socket udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # fetch dest_ip and dest_port from user dest_ip = input('Please enter remote IP: ') dest_port = int(input('Please enter remote Port: ')) # can only send bytes-like object, not str type # data_to_send = b'dinner tonight?' data_to_send = input('Enter your message: ') # send data to the destination # we need to convert string to bytes-like object by using str method encode() udp_socket.sendto(data_to_send.encode('utf-8'), (dest_ip, dest_port)) # receive data from the remote data_recv = udp_socket.recvfrom(1024) print(data_recv) # close the socket udp_socket.close()
In this program, we use the same socket to send and receive data.
After running this program, we enter the remote’s IP (or local IP if your Network Assistant is running on the same machine) and port. And then enter the message ‘hello’.
As you can see from the picture above, after you enter ‘hello’, the Net Assistant received it. The program then pauses and waits for receiving message from any program. In this case, we also use the same Net Assistant to send message ‘Hi, how are you!’ back to the program by just typing in the correct IP address and port in the right-bottom corner.
Because we didn’t bind port to our socket, so a random port number ‘54745’ is selected. Then click the ‘Send’ button, your program will receive the message and quit.
Let’s move on our full-fledged UDP chatter.
# UDP full fledged chatter import socket def send_data(udp_socket): dest_ip = input('Enter remote IP: ') # must convert port number to int dest_port = int(input('Enter remote port:')) msg_to_send = input('Enter message:') udp_socket.sendto(msg_to_send.encode('utf-8'), (dest_ip, dest_port)) def recv_data(udp_socket): data_recv = udp_socket.recvfrom(1024) message = data_recv[0].decode('utf-8') address = str(data_recv[1]) print('%s:%s' %(address, message)) def main(): # create a socket udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # bind local address to socket local_ip = '' local_port = 7878 udp_socket.bind((local_ip, local_port)) while True: print('-------UDP chatter--------') print('1: Send Mode') print('2: Receive Mode') print('0: Quit') # receive the mode from user mode = input('Please enter mode: ') if mode == '1': send_data(udp_socket) elif mode == '2': recv_data(udp_socket) elif mode == '0': break else: print('Wrong typing! Type again!') continue # close socket udp_socket.close() if __name__ == '__main__': main()
In this program, we create a udp_socket and bind it to port 7878. There are two modes for this program, ‘1’ for send mode, ‘2’ for receive mode. And ‘0’ for quit. We separate the details of sending and receiving messages from the main logic by defining two methods, send_data() and recv_data().
After running this program, we could send and receive data by choosing the right mode. By now, we haven’t talked about the multitasking, so our program cannot achieve sending and receiving at the same time. Later we will talk about multitasking.
Send mode
Receive Mode
Quit
By now, we have finished this UDP tutorial. Next tutorial we are gonna talk about TCP.
Comments
Post a Comment