Technical components

Version 7 (Paul Carensac, 04/20/2016 11:21 am)

1 1 Paul Carensac
h1. Technical components
2 2 Paul Carensac
3 4 Paul Carensac
Explanations on the technical components of the project : the ones we have created (internal), and the imported ones (external).
4 2 Paul Carensac
5 2 Paul Carensac
---
6 2 Paul Carensac
7 2 Paul Carensac
h2. %{margin-left:0px; font-weight:bold; font-size:25px;  display:block; color:red;}Internal components%
8 2 Paul Carensac
9 2 Paul Carensac
h3. Agent
10 1 Paul Carensac
11 4 Paul Carensac
The Agent class is in the common.agent.py file.
12 4 Paul Carensac
13 4 Paul Carensac
 * *I - Purpose*
14 4 Paul Carensac
15 4 Paul Carensac
    * Generically handles and creates the asynchronous modules
16 4 Paul Carensac
    * Uses the threading library (see below in External components) to make all modules independent
17 4 Paul Carensac
    * Provides an abstract class to be inherited
18 4 Paul Carensac
19 4 Paul Carensac
 * *II - Features*
20 4 Paul Carensac
21 4 Paul Carensac
    * Uses a config file (pyros_agent_config.ini) to set the network communication interface of all agents
22 4 Paul Carensac
    * Provides a 'work' method to override : this is the entry method of the newly created thread (see 'How to use it' section below)
23 4 Paul Carensac
    * Provides the 'receive' and 'analyze_message' methods to generically receive messages from network and analyze them 
24 4 Paul Carensac
25 4 Paul Carensac
 * *III - How to use it ?*
26 4 Paul Carensac
27 4 Paul Carensac
    Each of these points are +NECESSARY+
28 4 Paul Carensac
29 4 Paul Carensac
    * Create a new class that inherits from Agent
30 4 Paul Carensac
    * In the __init__ method, first call the __init__ method of Agent, passing the name of the agent as second parameter (they are defined in the Agent class, eg: Agent.SCHEDULER)
31 4 Paul Carensac
    * Inside the class, define the messages your agent can receive (eg: MSG_OBS_FINISHED = "Observation finished")
32 4 Paul Carensac
    * Create a method to be called for every message you created
33 4 Paul Carensac
    * In the __init__, after calling the Agent's __init__, associate each message to its associated function in the 'self.actions_by_message' dictionary (eg: self.actions_by_message[self.MSG_OBS_FINISHED] = self.observation_finished)
34 4 Paul Carensac
    * Override the method work : this will be the entry function of the new thread, so do whatever you need. This MUST NOT be an infinite loop, because Agent's receive method will be called after this one
35 4 Paul Carensac
    * If ever needed, override the 'shutdown' method, it will be called when your agent receive the Agent.SHUTDOWN message (eg: if you created another thread in the 'work' method, you need to close it)
36 5 Paul Carensac
    * To start the agent, just instantiate your class and do MyClass.start() (the 'work' method will be called)
37 1 Paul Carensac
38 5 Paul Carensac
    The main points to understand are that you can do whatever you want (but non-blocking) in work method (like creating new threads or variables' initialization), then the only entry points are the message-associated methods
39 4 Paul Carensac
40 7 Paul Carensac
 * *IV - Important : pyros agents launching*
41 7 Paul Carensac
42 7 Paul Carensac
    * In pyros, there is maximum 1 agent per application
43 7 Paul Carensac
    * The agent must be started at application start :
44 7 Paul Carensac
        
45 7 Paul Carensac
        * In the MyApp.apps.py file, create a class inheriting from django.apps.AppConfig
46 7 Paul Carensac
        * Define the 'name' attribute in it, giving it the name of the agent
47 7 Paul Carensac
        * Create a 'ready(self)' method
48 7 Paul Carensac
        * in the ready method, import your agent implementation, instantiate it and start it
49 7 Paul Carensac
<pre>
50 7 Paul Carensac
from django.apps import AppConfig
51 7 Paul Carensac
52 7 Paul Carensac
53 7 Paul Carensac
class AlertManagerConfig(AppConfig):
54 7 Paul Carensac
    name = 'alert_manager'
55 7 Paul Carensac
    
56 7 Paul Carensac
    def ready(self):
57 7 Paul Carensac
        from alert_manager.agent import AlertManagerAgent
58 7 Paul Carensac
        self.agent = AlertManagerAgent()
59 7 Paul Carensac
        self.agent.start()
60 7 Paul Carensac
</pre>
61 7 Paul Carensac
62 2 Paul Carensac
h3. Sender
63 1 Paul Carensac
64 4 Paul Carensac
The Sender class is in the common.sender.py file
65 4 Paul Carensac
66 4 Paul Carensac
 * *I - Purpose*
67 4 Paul Carensac
68 4 Paul Carensac
    * Send a given message to an agent
69 4 Paul Carensac
70 4 Paul Carensac
 * *II - Features*
71 4 Paul Carensac
72 4 Paul Carensac
    * Uses the 'pyros_agent_config.ini' file to get the agents' network interface configuration (ip and port)
73 4 Paul Carensac
    * Provide a 'send_to' static method to send the messages
74 4 Paul Carensac
75 4 Paul Carensac
 * *III - How to use it ?*
76 4 Paul Carensac
77 4 Paul Carensac
    * The targeted agent must be described in 'pyros_agent_config.ini'
78 4 Paul Carensac
    * Use Sender.send_to method, giving as first parameter the name of the targeted agent (eg: Agent.SCHEDULER), and as second parameter the message (eg: Agent.SHUTDOWN)
79 4 Paul Carensac
    * /!\ send_to is a static method, you don't need to instantiate a Sender (just do Sender.send_to(...))
80 2 Paul Carensac
---
81 2 Paul Carensac
82 2 Paul Carensac
h2. %{margin-left:0px; font-weight:bold; font-size:25px;  display:block; color:red;}External components%
83 2 Paul Carensac
84 1 Paul Carensac
h3. Threading library
85 5 Paul Carensac
86 5 Paul Carensac
 * *I - Purpose*
87 5 Paul Carensac
88 5 Paul Carensac
    * Simply create threads with basic communication
89 5 Paul Carensac
    * Allows to handle concurrent access
90 5 Paul Carensac
91 5 Paul Carensac
 * *II - Features*
92 5 Paul Carensac
93 5 Paul Carensac
    Provides :
94 5 Paul Carensac
95 5 Paul Carensac
    * A Thread class to inherit from, with a run() method that will be called when the thread starts
96 5 Paul Carensac
    * An Event class to set/unset a boolean in order to transmit message to the thread
97 5 Paul Carensac
    * Lock and RLock object to handle concurrent access
98 5 Paul Carensac
99 5 Paul Carensac
 * *III - How to use it ?*
100 5 Paul Carensac
101 5 Paul Carensac
    <pre>from threading import Thread, Event</pre>
102 5 Paul Carensac
103 5 Paul Carensac
    * Thread
104 5 Paul Carensac
105 5 Paul Carensac
        * Create a class inheriting from Thread
106 5 Paul Carensac
        * Override 'run' method, that will be called at thread start
107 5 Paul Carensac
        * Instantiate your class, and do MyClass.start() to create the thread
108 5 Paul Carensac
109 5 Paul Carensac
    * Event
110 5 Paul Carensac
111 5 Paul Carensac
        * Create an Event variable in your Thread-inheriting class (eg: 'self.stop_event = Event()')
112 5 Paul Carensac
        * After thread starts, you can set/unset the event by doing MyClass.stop_event.set() / .clear()
113 5 Paul Carensac
        * There are a few useful methods, see this link for further information : https://docs.python.org/3/library/threading.html#threading.Event
114 5 Paul Carensac
    * Lock / RLock
115 5 Paul Carensac
116 5 Paul Carensac
        * Still not used, see online documentation : https://docs.python.org/3/library/threading.html#lock-objects
117 2 Paul Carensac
118 2 Paul Carensac
h3. Socket library
119 2 Paul Carensac
120 6 Paul Carensac
 * *I - Purpose*
121 6 Paul Carensac
122 6 Paul Carensac
    * Handle network communication, just giving IP and Port of the interlocutors
123 6 Paul Carensac
124 6 Paul Carensac
 * *II - Features*
125 6 Paul Carensac
126 6 Paul Carensac
    * 'server' system to create an interface, waiting for client connections and sending / receiving data from them
127 6 Paul Carensac
    * 'client' system to connect to a server, and send/receive data from it
128 6 Paul Carensac
129 6 Paul Carensac
 * *III - How to use it ?*
130 6 Paul Carensac
131 6 Paul Carensac
    * Server
132 6 Paul Carensac
133 6 Paul Carensac
        * Instantiate socket and wait for connections
134 6 Paul Carensac
<pre>
135 6 Paul Carensac
self.server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)   # create the socket
136 6 Paul Carensac
self.server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) # for the port to be immediately re-usable after closing the socket
137 6 Paul Carensac
self.server_socket.bind((self.ip, self.receive_port))                    # associate the socket to an ip and a port
138 6 Paul Carensac
self.server_socket.listen(12)                                            # wait for connections (here, 12 connections can be simultaneously waiting for acceptance)
139 6 Paul Carensac
</pre>
140 6 Paul Carensac
        * Accept connections
141 6 Paul Carensac
<pre>
142 6 Paul Carensac
conn, addr = self.server_socket.accept() # conn is a new socket created at the connection
143 6 Paul Carensac
</pre>
144 6 Paul Carensac
        * Exchanging messages
145 6 Paul Carensac
<pre>
146 6 Paul Carensac
conn.send(bytes(message, 'UTF-8'))          # sending
147 6 Paul Carensac
data = conn.recv(self.buffer_size).decode() # receiving
148 6 Paul Carensac
</pre>
149 6 Paul Carensac
        * Closing sockets when you're done with them
150 6 Paul Carensac
<pre>
151 6 Paul Carensac
conn.close()
152 6 Paul Carensac
...
153 6 Paul Carensac
server_socket.close()
154 6 Paul Carensac
</pre>
155 6 Paul Carensac
156 6 Paul Carensac
    * Client
157 6 Paul Carensac
158 6 Paul Carensac
        * Instantiate the socket and connect to a server
159 6 Paul Carensac
<pre>
160 6 Paul Carensac
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
161 6 Paul Carensac
client_socket.connect((dest_ip, dest_receive_port))
162 6 Paul Carensac
</pre>
163 6 Paul Carensac
        * Exchanging messages
164 6 Paul Carensac
<pre>
165 6 Paul Carensac
client_socket.send(bytes(message, 'UTF-8'))          # sending
166 6 Paul Carensac
data = client_socket.recv(self.buffer_size).decode() # receiving
167 6 Paul Carensac
</pre>
168 6 Paul Carensac
        * Closing sockets when you're done with them
169 6 Paul Carensac
<pre>
170 6 Paul Carensac
client_socket.close()
171 6 Paul Carensac
</pre>
172 6 Paul Carensac
173 2 Paul Carensac
---