PYROS

h1. TODO

List of tasks by application, and general tasks (organization, tools, ...)

{{>toc}}

---

h2. %{margin-left:0px; font-weight:bold; font-size:25px;  display:block; color:red;}General tasks%

* Pyros : 

 * Doit pouvoir être démarré indifféremment AVANT (par défaut) les devices, ou APRES (actuellement, le AVANT est TODO)

 * De manière générale, doit être le plus possible "tolérant aux pannes"

 * Doit être le plus générique et donc paramétrable possible

* Django : 

 * Upgrade to Django 1.10

 * Remplacer le serveur web de "dev" (manage runserver) par un vrai serveur web pour la "prod" (Apache pour les fichiers statiques + serveur d'application Python pour le code Python, par exemple gunicorn)

(cf https://projects.irap.omp.eu/projects/pyros/wiki/Project_Development#Serveur-Web)

* Comet installation : Include in the install_requirements.sh script (by using stable official version for Python 3 (when available) on Linux (and Mac) and Windows 10

* git : Create a "dev" branch (do not write anymore on the "master" branch)

* Simulators : 

 * on doit se rendre compte qu'ils sont "vivants" (ils seront progressivement remplacés par les vrais devices, mais resteront toujours utilisables à leur place)

  * Le Monitoring doit les interroger régulièrement sur leur statut (check_status, DONE)

  * Il doit stocker les statuts dans la BD (TODO)

  * Ces statuts doivent être affichés au fur et à mesure sur la page web du "device" correspondant (Devices, Site, Weather) (TODO)

 * quels sont les devices bloquants et non bloquants ?

  * Bloquants: si indisponibles (en panne ou pas démarrés), Pyros met en pause le sous-système d'observation

Seules 3 fonctions restent toujours actives: alertes, monitoring, et serveur web (pour les routines et la surveillance)

   * Telescope

   * Camera VIS (???)

   * PLC (weather + observation conditions + site)

  * Non bloquants: si indisponibles (en panne ou pas démarrés), Pyros se contente d'en prendre note, et continue son fonctionnement normal tout en restant dans l'attente de leur redémarrage pour les réintégrer

   * Camera VIS (???)

   * Camera NIR

* Users: gérer les 3 profils (admin, expert, user)

* Doc :

 * TEST:

  * Unit tests : test of each module

  * Functional tests (with Celery and simulators): complete workflow

 * LAUNCH:

  * Start simulators

  * Start Celery

  * Start pyros

 * USE:

  * Administration of the database: http://localhost:8000/admin

  * Interacting with Pyros: http://localhost:8000

   * Watch the environment: Devices, Site, Weather

   * Watch the schedule: Schedule

   * Watch the data processing (workflow): System (Dashboard)

   * Submit a Routine Request (and get results): Routines

   * Watch alerts: Alerts

   * Simulate an Alert: Alerts (TODO: Un user "admin" doit pouvoir déclencher une alerte type depuis la page web Alerts)

   * Manual operations on the Telescope : Devices/Telescope (TODO)

   * Manage users : Users

---

h2. %{margin-left:0px; font-weight:bold; font-size:25px;  display:block; color:red;}Applications tasks%

h3. Dashboard

 * Create the backoffice views as the modules are integrated in pyros

 * Think about a system of permissions

h3. Scheduler

 * views

  * Link the main page to the current schedule instead of the simulation page

  * Show user sequences in the schedules (with links)

  * Give accces to old schedules (day / days before / all end-night plannings / all plannings)

  * Give access to the refused sequences of a schedule, and the reasons of rejects

 * scheduler

  * Change the system to determine night start/end (they must be given in parameter, only if first_schedule is True)

  * Store the reasons of rejects (create a new attribute, in shs ?)

  * What is the 'flag' attribute ? (@AK)

  * Do not create the execute_sequence tasks if it's not the night (- 120 seconds)

  * Priority and quota computing

  * Quotas evolution

  * Blank space filling

  * At the end of a scheduling send, it to the IC ?

h3. Alert Manager

 * Web :

  * Print if there is an alert in progress in the main page

  * Link the alerts to their status and results

 * Connect to a real VOEvent broker

 * Determine the communication with FSC for strategy change

 * VOEvents :

  * Extract the good fields (see AK Q&A 07/01/2016)

  * Fill the request & alerts objects

  * Use strategies to build a request

  * Possibility to change the default strategy

  * Handle VOEvent updates

  * Be careful to not create 2 alerts for a same GRB, seen by 2 different satellites 

h3. Analyzer

 * Apply the calibrations in the right function

 * Apply the analyses only if it's a GRB

 * Implement the analyses

 * Send analyses to FSC

h3. Majordome

 * TaskManager

  * When a sequence is cancelled, give back the quota to the user

  * In case of alert, do not stop the ongoing plan, and make the instruments abort

 * execute_sequence

  * Add the PLC checks at start (to see if we do the slew)

  * Use the global telescope (instead of creating one here)

  * Give first_schedule as false when a scheduling is launched

  * Remove the default countdown (1, for tests)

 * system_pause

  * Abort the isntruments

  * Stop the execution tasks

 * system_restart

  * Start a scheduling

 * change_obs_conditions

  * Change sequences status (if  needed)

  * If some status changed, re-launch a scheduling

h3. Monitoring

 * views

  * Move the dashboard here

  * Print the instrument status

  * Print PLC informations (with the evolution)

  * In the dashboard screens, put scroll on each screen to see the old logs

 * Monitoring task

  * Uncomment the scheduling at the beginning

  * Implement night start/end computation

  * Initialize communication with the instruments

  * Configure intruments at start

  * Send software versions to the IC

  * Initialize connection with PLC

  * After the starting actions, loop to wait for the instruments configuration to be finished

  * Ask PLC for status

  * Ask filter wheel for status

  * Store the instruments & PLC status

  * Send all status to IC

  * Analyse PLC status (obs conditions, ...)

  * Create tasks of obs condition changes

  * At the beginning of the loop, ping the DISABLED devices. If they answer, change their status (TBD), and do their initialization (ex: If it's a camera, set its temperature to -150°)

h3. Observation Manager

 * execute_plan :

  * Use the cameras at a global level instead of creating them here (same for the filter wheels)

  * Uncomment the instruments_ready waiting function

  * Uncomment the observation_ending waiting function

  * Try to remove code duplication

  * Determine what needs to be done at the end of an observation

 * create_calibrations :

  * Make the calibration images

  * Generate super images

  * Send them to the IC

h3. Routine Manager

 * Web

  * Put the goods fields (for coordinates etc)

  * Only propose the objects that matches the conditions (ex: scientific programs of the user only)

  * Do all the needed checks

  * Add automatic computation of JD1/JD2

  * Add checkbox for JD / GD

  * Add options : copy my sequence on x days, and authorise report

  * Add ETC-IS simulation

  * Add help for new users (and for it the first time an account come on the page)

 * Do more checks at unserialization

 * views

  * When saving, do more checks on coordinates, jd1/2 etc

  * Uncomment filter for alerts removing

  * When submitting, use the monitoring to determine sequences status

  * When submitting, modify the first_schedule to False, when scheduling

  * When unsubmitting, uncomment the check for EXED and EXING removing

  * When unsubmitting, uncomment the scheduling and change the first_schedule to False

h3. User Manager

 * Password recovery

 * Profile page

 * User validation by administrator / commission

 * Handle permissions and access

h3. Common

 * Change the 'first_schedule' to False at the end of RequestBuilder.validate()

h3. Devices

 * When a socket connection fails (exception thrown), set the device to UNACTIVATED (or that kind of status) in the DB, and create a system_pause task (majordome)