PYROS

h1. Pyros applications

List and details of all the pyros applications.

{{>toc}}

---

h2. %{margin-left:0px; font-weight:bold; font-size:25px;  display:block; color:red;}pyrosapp%

h3. Purpose

 * Contains all the database Models

 * Basic tests in tests.py

 * Backoffice configuration in admin.py

h3. Notes

 * Should be removed, and the models moved in the "common" application

---

h2. %{margin-left:0px; font-weight:bold; font-size:25px;  display:block; color:red;}dashboard%

h3. Purpose

 * Interface for all external users

 * Leads to displays and actions for all the pyros modules (users, requests, system execution, ...)

h3. Evolution

 * Creating application

    * manage.py startapp dashboard

    * added 'dashboard' in settings.py -> INSTALLED_APPS

    * created a urls.py file in dashboard module

    * added _url(r'^dashboard/', include('dashboard.urls'))_ in pyros/urls.py -> urlpatterns

    * created templates/ and templates/dashboard/ folders in dashboard module

 * Main page

    * added 'home' view in views.py

    * linked 'home' view to 8000/dashboard URL

    * created a template for the homepage in templates/dashboard/ (with bootstrap)

    * created views, views linking (urls.py) and templates for the dashboard modules

    * redirected mainpage buttons to Admin interface (temporary)

 * System page (logs)

    * Retrieve logs from the db

    * Print logs and automatically update every second via an ajax request

h3. Notes

 * The buttons lead to the Admin interface for the moment

 * Added bootstrap3 module (see Installation)

 * Added Django Template Editor (see Eclipse configuration)

---

h2. %{margin-left:0px; font-weight:bold; font-size:25px;  display:block; color:red;}scheduler%

h3. Purpose

 * Creates the planning with the OBSERVABLE sequences

 * Give access to a web page to see the current & old plannings

h3. Evolution

 * Creating application

    * manage.py startapp scheduler

    * added 'scheduler' in settings.py -> INSTALLED_APPS

    * created a urls.py file in scheduler module

    * added _url(r'^scheduler/', include('scheduler.urls'))_ in pyros/urls.py -> urlpatterns

    * created templates/ and templates/scheduler/ folders in scheduler module

 * Model modifications

    * Schedule

        * Remove day_start

        * Remove day_stop

        * Add plan_start

        * Add plan_end

        * Enum system for the status 

    * ScheduleHistory

        * Remove day_start

        * Remove day_stop

        * Add plan_start

        * Add plan_end

    * Sequence

        * Remove exec_start

        * Remove exec_stop

        * Add tsp

        * Add tep

        * Add jd1

        * Add jd2

        * Add deltaTL

        * Add deltaTR

        * Add t_prefered

        * Changed duration from Float to DecimalField (more precise)

        * Add overhead

    * manage.py makemigrations sheduler ; manage.py migrate

 * Creation of Scheduler and Interval classes in models.py

 * Implementation of the Interval class

 * Implementation of the Scheduler's 'make_schedule' function (and children). This is the only entry point for now. This function creates the planning (organizes the observable sequences).

 * Creation of the web interface

    * Added current_schedule.html in template/scheduler folder

    * Created view and url linking to this template (with current planning retrieving)

 * Creation of the simulator

    * Created a second entry point in the Scheduler class (with a few minor adaptations to handle SIMULATION mode)

    * Created a simulator module in the scheduler

        * Added the MyHTMLParser class (easy implementation of HTMLParser)

        * Added Simulator class. It parses a file given in parametr to retrieve sequences and create a schedule

    * Created a second view linked to schedule/simulation to show simulation results

 * Supression du système d'agent et ajout de la tâche (celery) de scheduling

 * Main update 19/05/2016 : overheads & historic system

    * Deleted ScheduleHistory

    * Transformed the Sequence - Schedule relation to ManyToMany

    * Moved the scheduling information into the M2M relationship class (shs = ScheduleHasSequences)

    * Added a static overhead into scheduling (can be set from the calling code, through scheduler.max_overhead = blabla(float))

h3. Tests

 * Some tests in test.py to test the different functionalities of the scheduler. These tests include the simulation.

 * A simulator at localhost:8000/scheduler/simulation, taking sequences from scheduler/sequences_cador.html

h3. Notes

 * Priorities and quotas are default-calculated (for the moment)

 * What is the 'flag' attribute in the Schedule model ?

---

h2. %{margin-left:0px; font-weight:bold; font-size:25px;  display:block; color:red;}alert_manager%

h3. Purpose

 * Listen to VOEvent network

 * Sort interesting events

 * Create requests according to these events

 * Manage the requests created via the alerts

 * Change the observation strategies for alerts

h3. Evolution

 * Creating application

    * See scheduler documentation above

 * Implementation of the agent system

    * Created agent.py whith a "class AlertManagerAgent(Agent):" inside

    * Overriding agent's 'work' and 'shutdown' method

    * Added message(s) to agent.actions_by_message dictionnary, with the functions associated to these messages

    * Added the Agent's launch system in apps.py

 * Implementation of the VOEventListener Thread

    * Created a "class VOEventListener(Thread):" in agent.py

    * Overriding the run method

 * 19/05/2016 : deleted the agent system (deleted from the applications below too)

 * Added some VOEvents in events_to_send

 * Installed Comet (see in [[Technical components]] section)

 * Created 2 celery tasks in tesks.py : alert_listener and change_obs_strategy

 * Added alert_listener launch at server start

 * Implemented alert_listener : it waits for an event file to be created in events_received, then read it and creates a request

 * Added a simulator system ( see section Tests below )

 * Added the strategy_change page

 * Removed launching at start : it is now started by the monitoring task

h3. Tests

 * Simulator : 

  * You first need to set settings.SIMULATION to True (in pyros/settings.py)

  * Start celery workers (scripts/start_celery_workers.sh)

  * Start server (manage.py runserver)

  * Go to localhost:8000/alert_manager/simulation, and press the button !

  * This will read into the simulation_sequences file.

 * Test AlertListenerTestsCelery.test_alert_reception :

  * Tests if the alert_listener is working well.

  * +Be careful :+ this will act on the production database, and will start the entire alert workflow

  * To start it : script/celery_test.sh alert_manager

 * Tests TestStrategyChange.* :

  * Test the changing of the observation strategy with errors and different parameters

h3. Notes

 * The request created by alert_listener is hard-coded

---

h2. %{margin-left:0px; font-weight:bold; font-size:25px;  display:block; color:red;}analyzer%

h3. Purpose

 * Apply calibration files to the images taken by the observation manager

 * Analyze the images produced by the calibrations

h3. Evolution

 * Creating application

    * See scheduler documentation above

 * Implemented the skeleton of the module in the task Analysis (call  a function for calibrations then a function for analyses)

h3. Notes

---

h2. %{margin-left:0px; font-weight:bold; font-size:25px;  display:block; color:red;}majordome%

h3. Purpose

 * Handles the execution of the sequences in the right order during the night

 * Handles the system start & stop

 * Handles the changes of observation conditions

 * Stops the celery tasks when needed

h3. Evolution

 * Creating application

  * See scheduler documentation above

  * Created all the tasks in tasks.py

  * Created the TaskManager to stop the sequence & plan executions when an alert / routine is received

  * Implemented execute_sequence (not finished)

h3. Notes

---

h2. %{margin-left:0px; font-weight:bold; font-size:25px;  display:block; color:red;}monitoring%

h3. Purpose

 * It is the only tasks launched at the system's start

 * Make all the starting actions : instruments configuration, connection to the PLC, updating software versions, determine night start/end & start a scheduling

 * Starts the alert_listener

 * Infinite loop to regularly check the instruments and PLC status (and send them to the IC)

 * Handles night start & end

h3. Evolution

 * Creating application

    * See scheduler documentation above

 * System to launch the task at the system's start

 * Implement skeleton of task

 * Implement starting actions (with some empty functions)

 * Main loop to check PLC, instruments and night start / end (not finished)

h3. Notes

---

h2. %{margin-left:0px; font-weight:bold; font-size:25px;  display:block; color:red;}observation_manager%

h3. Purpose

 * Give orders the instruments to execute the plans

 * Take the calibration images and computes the 'super' calibration images

h3. Evolution

 * Creating application

    * See scheduler documentation above

 * Created the tasks : execute_plan_(nir/vis) & create_calibrations

 * Implemented the execute_plan tasks with some incomplete functions

h3. Notes

---

h2. %{margin-left:0px; font-weight:bold; font-size:25px;  display:block; color:red;}routine_manager%

h3. Purpose

 * Enables astronomers to create routines via the website

 * Import / export of routines (XML)

 * Display observation results for routines

h3. Evolution

 * Creating application

    * See scheduler documentation above

 * Created forms online for routine creation

 * Added import / export

 * Added routine submit / unsubmit

h3. Tests

 * Import fake file

 * Import invalid file

 * Import good file

 * Export incomplete

 * Export valid

h3. Notes

---

h2. %{margin-left:0px; font-weight:bold; font-size:25px;  display:block; color:red;}user_manager%

h3. Purpose

 * Handle user create / delete / login / logout

 * Allow to see user profile and make password recovery

h3. Evolution

 * Creating application

    * See scheduler documentation above

 * Taking and adapting templates from Alexandru project

 * User creation

  * Form in admin.py

  * Functions to verify if email doesn't exist and if passwords match

  * Added check to pass login page if user is already autheticated

  * Restricted access to all pyros views to authenticated users (except from user creation and login) with @login_required

h3. Tests

 * Creation

 * Login

 * wrong email

 * wrong password

 * logout

h3. Notes

---

h2. %{margin-left:0px; font-weight:bold; font-size:25px;  display:block; color:red;}common%

h3. Purpose

 * Regroups common data and functions shared by applications

h3. Content

 * RequestBuilder - class to build and save in DB the requests

  * You can start a request, then add sequences, albums and plans, given their parent class (ex: you need to give a sequence id to create an album)

  * Each function returns the ID of the element it created (to give it to the children classes)

  * To save a request, you need to use validate_request

 * Device and children

  * Implementation of devices communication, messages and initialization

  * All classes inherit from the Device class

  * All classes re-implement the list of messages

h3. Tests

 * Test the creation of a full request by the RequestBuilder

h3. Notes

 * Will probably contain the DB models at last ...