Technical components

Version 9 (Paul Carensac, 04/20/2016 11:21 am)

1 1 Paul Carensac
h1. Technical components
2 2 Paul Carensac
3 8 Paul Carensac
Explanations about the technical components of the project : the ones we have created (internal), and the imported ones (external).
4 2 Paul Carensac
5 2 Paul Carensac
---
6 2 Paul Carensac
7 2 Paul Carensac
h2. %{margin-left:0px; font-weight:bold; font-size:25px;  display:block; color:red;}Internal components%
8 2 Paul Carensac
9 2 Paul Carensac
h3. Agent
10 1 Paul Carensac
11 4 Paul Carensac
The Agent class is in the common.agent.py file.
12 4 Paul Carensac
13 4 Paul Carensac
 * *I - Purpose*
14 4 Paul Carensac
15 4 Paul Carensac
    * Generically handles and creates the asynchronous modules
16 4 Paul Carensac
    * Uses the threading library (see below in External components) to make all modules independent
17 4 Paul Carensac
    * Provides an abstract class to be inherited
18 4 Paul Carensac
19 4 Paul Carensac
 * *II - Features*
20 4 Paul Carensac
21 4 Paul Carensac
    * Uses a config file (pyros_agent_config.ini) to set the network communication interface of all agents
22 4 Paul Carensac
    * Provides a 'work' method to override : this is the entry method of the newly created thread (see 'How to use it' section below)
23 4 Paul Carensac
    * Provides the 'receive' and 'analyze_message' methods to generically receive messages from network and analyze them 
24 4 Paul Carensac
25 4 Paul Carensac
 * *III - How to use it ?*
26 4 Paul Carensac
27 4 Paul Carensac
    Each of these points are +NECESSARY+
28 4 Paul Carensac
29 4 Paul Carensac
    * Create a new class that inherits from Agent
30 4 Paul Carensac
    * In the __init__ method, first call the __init__ method of Agent, passing the name of the agent as second parameter (they are defined in the Agent class, eg: Agent.SCHEDULER)
31 4 Paul Carensac
    * Inside the class, define the messages your agent can receive (eg: MSG_OBS_FINISHED = "Observation finished")
32 4 Paul Carensac
    * Create a method to be called for every message you created
33 4 Paul Carensac
    * In the __init__, after calling the Agent's __init__, associate each message to its associated function in the 'self.actions_by_message' dictionary (eg: self.actions_by_message[self.MSG_OBS_FINISHED] = self.observation_finished)
34 4 Paul Carensac
    * Override the method work : this will be the entry function of the new thread, so do whatever you need. This MUST NOT be an infinite loop, because Agent's receive method will be called after this one
35 4 Paul Carensac
    * If ever needed, override the 'shutdown' method, it will be called when your agent receive the Agent.SHUTDOWN message (eg: if you created another thread in the 'work' method, you need to close it)
36 5 Paul Carensac
    * To start the agent, just instantiate your class and do MyClass.start() (the 'work' method will be called)
37 1 Paul Carensac
38 5 Paul Carensac
    The main points to understand are that you can do whatever you want (but non-blocking) in work method (like creating new threads or variables' initialization), then the only entry points are the message-associated methods
39 4 Paul Carensac
40 7 Paul Carensac
 * *IV - Important : pyros agents launching*
41 7 Paul Carensac
42 7 Paul Carensac
    * In pyros, there is maximum 1 agent per application
43 7 Paul Carensac
    * The agent must be started at application start :
44 7 Paul Carensac
        
45 7 Paul Carensac
        * In the MyApp.apps.py file, create a class inheriting from django.apps.AppConfig
46 7 Paul Carensac
        * Define the 'name' attribute in it, giving it the name of the agent
47 7 Paul Carensac
        * Create a 'ready(self)' method
48 7 Paul Carensac
        * in the ready method, import your agent implementation, instantiate it and start it
49 7 Paul Carensac
<pre>
50 7 Paul Carensac
from django.apps import AppConfig
51 7 Paul Carensac
52 7 Paul Carensac
53 7 Paul Carensac
class AlertManagerConfig(AppConfig):
54 7 Paul Carensac
    name = 'alert_manager'
55 7 Paul Carensac
    
56 7 Paul Carensac
    def ready(self):
57 7 Paul Carensac
        from alert_manager.agent import AlertManagerAgent
58 7 Paul Carensac
        self.agent = AlertManagerAgent()
59 7 Paul Carensac
        self.agent.start()
60 7 Paul Carensac
</pre>
61 7 Paul Carensac
62 2 Paul Carensac
h3. Sender
63 1 Paul Carensac
64 4 Paul Carensac
The Sender class is in the common.sender.py file
65 4 Paul Carensac
66 4 Paul Carensac
 * *I - Purpose*
67 4 Paul Carensac
68 4 Paul Carensac
    * Send a given message to an agent
69 4 Paul Carensac
70 4 Paul Carensac
 * *II - Features*
71 4 Paul Carensac
72 4 Paul Carensac
    * Uses the 'pyros_agent_config.ini' file to get the agents' network interface configuration (ip and port)
73 4 Paul Carensac
    * Provide a 'send_to' static method to send the messages
74 4 Paul Carensac
75 4 Paul Carensac
 * *III - How to use it ?*
76 4 Paul Carensac
77 4 Paul Carensac
    * The targeted agent must be described in 'pyros_agent_config.ini'
78 4 Paul Carensac
    * Use Sender.send_to method, giving as first parameter the name of the targeted agent (eg: Agent.SCHEDULER), and as second parameter the message (eg: Agent.SHUTDOWN)
79 4 Paul Carensac
    * /!\ send_to is a static method, you don't need to instantiate a Sender (just do Sender.send_to(...))
80 9 Paul Carensac
81 2 Paul Carensac
---
82 2 Paul Carensac
83 2 Paul Carensac
h2. %{margin-left:0px; font-weight:bold; font-size:25px;  display:block; color:red;}External components%
84 2 Paul Carensac
85 1 Paul Carensac
h3. Threading library
86 5 Paul Carensac
87 5 Paul Carensac
 * *I - Purpose*
88 5 Paul Carensac
89 5 Paul Carensac
    * Simply create threads with basic communication
90 5 Paul Carensac
    * Allows to handle concurrent access
91 5 Paul Carensac
92 5 Paul Carensac
 * *II - Features*
93 5 Paul Carensac
94 5 Paul Carensac
    Provides :
95 5 Paul Carensac
96 5 Paul Carensac
    * A Thread class to inherit from, with a run() method that will be called when the thread starts
97 5 Paul Carensac
    * An Event class to set/unset a boolean in order to transmit message to the thread
98 5 Paul Carensac
    * Lock and RLock object to handle concurrent access
99 5 Paul Carensac
100 5 Paul Carensac
 * *III - How to use it ?*
101 5 Paul Carensac
102 5 Paul Carensac
    <pre>from threading import Thread, Event</pre>
103 5 Paul Carensac
104 5 Paul Carensac
    * Thread
105 5 Paul Carensac
106 5 Paul Carensac
        * Create a class inheriting from Thread
107 5 Paul Carensac
        * Override 'run' method, that will be called at thread start
108 5 Paul Carensac
        * Instantiate your class, and do MyClass.start() to create the thread
109 5 Paul Carensac
110 5 Paul Carensac
    * Event
111 5 Paul Carensac
112 5 Paul Carensac
        * Create an Event variable in your Thread-inheriting class (eg: 'self.stop_event = Event()')
113 5 Paul Carensac
        * After thread starts, you can set/unset the event by doing MyClass.stop_event.set() / .clear()
114 5 Paul Carensac
        * There are a few useful methods, see this link for further information : https://docs.python.org/3/library/threading.html#threading.Event
115 5 Paul Carensac
    * Lock / RLock
116 5 Paul Carensac
117 5 Paul Carensac
        * Still not used, see online documentation : https://docs.python.org/3/library/threading.html#lock-objects
118 2 Paul Carensac
119 2 Paul Carensac
h3. Socket library
120 2 Paul Carensac
121 6 Paul Carensac
 * *I - Purpose*
122 6 Paul Carensac
123 6 Paul Carensac
    * Handle network communication, just giving IP and Port of the interlocutors
124 6 Paul Carensac
125 6 Paul Carensac
 * *II - Features*
126 6 Paul Carensac
127 6 Paul Carensac
    * 'server' system to create an interface, waiting for client connections and sending / receiving data from them
128 6 Paul Carensac
    * 'client' system to connect to a server, and send/receive data from it
129 6 Paul Carensac
130 6 Paul Carensac
 * *III - How to use it ?*
131 6 Paul Carensac
132 6 Paul Carensac
    * Server
133 6 Paul Carensac
134 6 Paul Carensac
        * Instantiate socket and wait for connections
135 6 Paul Carensac
<pre>
136 6 Paul Carensac
self.server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)   # create the socket
137 6 Paul Carensac
self.server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) # for the port to be immediately re-usable after closing the socket
138 6 Paul Carensac
self.server_socket.bind((self.ip, self.receive_port))                    # associate the socket to an ip and a port
139 6 Paul Carensac
self.server_socket.listen(12)                                            # wait for connections (here, 12 connections can be simultaneously waiting for acceptance)
140 6 Paul Carensac
</pre>
141 6 Paul Carensac
        * Accept connections
142 6 Paul Carensac
<pre>
143 6 Paul Carensac
conn, addr = self.server_socket.accept() # conn is a new socket created at the connection
144 6 Paul Carensac
</pre>
145 6 Paul Carensac
        * Exchanging messages
146 6 Paul Carensac
<pre>
147 6 Paul Carensac
conn.send(bytes(message, 'UTF-8'))          # sending
148 6 Paul Carensac
data = conn.recv(self.buffer_size).decode() # receiving
149 6 Paul Carensac
</pre>
150 6 Paul Carensac
        * Closing sockets when you're done with them
151 6 Paul Carensac
<pre>
152 6 Paul Carensac
conn.close()
153 6 Paul Carensac
...
154 6 Paul Carensac
server_socket.close()
155 6 Paul Carensac
</pre>
156 6 Paul Carensac
157 6 Paul Carensac
    * Client
158 6 Paul Carensac
159 6 Paul Carensac
        * Instantiate the socket and connect to a server
160 6 Paul Carensac
<pre>
161 6 Paul Carensac
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
162 6 Paul Carensac
client_socket.connect((dest_ip, dest_receive_port))
163 6 Paul Carensac
</pre>
164 6 Paul Carensac
        * Exchanging messages
165 6 Paul Carensac
<pre>
166 6 Paul Carensac
client_socket.send(bytes(message, 'UTF-8'))          # sending
167 6 Paul Carensac
data = client_socket.recv(self.buffer_size).decode() # receiving
168 6 Paul Carensac
</pre>
169 6 Paul Carensac
        * Closing sockets when you're done with them
170 6 Paul Carensac
<pre>
171 6 Paul Carensac
client_socket.close()
172 6 Paul Carensac
</pre>
173 6 Paul Carensac
174 2 Paul Carensac
---