History

Wiki » Project Development »

« Previous - Version 59/60 (diff) - Next » - Current version
Paul Carensac, 08/09/2016 04:44 pm

Pyros applications

List and details of all the pyros applications.

dashboard

Purpose

  • Interface for all external users
  • Leads to displays and actions for all the pyros modules (users, requests, system execution, ...)

Evolution

  • Creating application
    • manage.py startapp dashboard
    • added 'dashboard' in settings.py -> INSTALLED_APPS
    • created a urls.py file in dashboard module
    • added url(r'^dashboard/', include('dashboard.urls')) in pyros/urls.py -> urlpatterns
    • created templates/ and templates/dashboard/ folders in dashboard module
  • Main page
    • added 'home' view in views.py
    • linked 'home' view to 8000/dashboard URL
    • created a template for the homepage in templates/dashboard/ (with bootstrap)
    • created views, views linking (urls.py) and templates for the dashboard modules
    • redirected mainpage buttons to Admin interface (temporary)
  • System page (logs)
    • Retrieve logs from the db
    • Print logs and automatically update every second via an ajax request

Notes

  • The buttons lead to the Admin interface for the moment
  • Added bootstrap3 module (see Installation)
  • Added Django Template Editor (see Eclipse configuration)

scheduler

Purpose

  • Creates the planning with the OBSERVABLE sequences
  • Give access to a web page to see the current & old plannings

Evolution

  • Creating application
    • manage.py startapp scheduler
    • added 'scheduler' in settings.py -> INSTALLED_APPS
    • created a urls.py file in scheduler module
    • added url(r'^scheduler/', include('scheduler.urls')) in pyros/urls.py -> urlpatterns
    • created templates/ and templates/scheduler/ folders in scheduler module
  • Model modifications
    • Schedule
      • Remove day_start
      • Remove day_stop
      • Add plan_start
      • Add plan_end
      • Enum system for the status
    • ScheduleHistory
      • Remove day_start
      • Remove day_stop
      • Add plan_start
      • Add plan_end
    • Sequence
      • Remove exec_start
      • Remove exec_stop
      • Add tsp
      • Add tep
      • Add jd1
      • Add jd2
      • Add deltaTL
      • Add deltaTR
      • Add t_prefered
      • Changed duration from Float to DecimalField (more precise)
      • Add overhead
    • manage.py makemigrations sheduler ; manage.py migrate
  • Creation of Scheduler and Interval classes in models.py
  • Implementation of the Interval class
  • Implementation of the Scheduler's 'make_schedule' function (and children). This is the only entry point for now. This function creates the planning (organizes the observable sequences).
  • Creation of the web interface
    • Added current_schedule.html in template/scheduler folder
    • Created view and url linking to this template (with current planning retrieving)
  • Creation of the simulator
    • Created a second entry point in the Scheduler class (with a few minor adaptations to handle SIMULATION mode)
    • Created a simulator module in the scheduler
      • Added the MyHTMLParser class (easy implementation of HTMLParser)
      • Added Simulator class. It parses a file given in parametr to retrieve sequences and create a schedule
    • Created a second view linked to schedule/simulation to show simulation results
  • Supression du système d'agent et ajout de la tâche (celery) de scheduling
  • Main update 19/05/2016 : overheads & historic system
    • Deleted ScheduleHistory
    • Transformed the Sequence - Schedule relation to ManyToMany
    • Moved the scheduling information into the M2M relationship class (shs = ScheduleHasSequences)
    • Added a static overhead into scheduling (can be set from the calling code, through scheduler.max_overhead = blabla(float))

Tests

  • Some tests in test.py to test the different functionalities of the scheduler. These tests include the simulation.
  • A simulator at localhost:8000/scheduler/simulation, taking sequences from scheduler/sequences_cador.html

Notes

  • Priorities and quotas are default-calculated (for the moment)
  • What is the 'flag' attribute in the Schedule model ?

alert_manager

Purpose

  • Listen to VOEvent network
  • Sort interesting events
  • Create requests according to these events
  • Manage the requests created via the alerts
  • Change the observation strategies for alerts

Evolution

  • Creating application
    • See scheduler documentation above
  • Implementation of the agent system
    • Created agent.py whith a "class AlertManagerAgent(Agent):" inside
    • Overriding agent's 'work' and 'shutdown' method
    • Added message(s) to agent.actions_by_message dictionnary, with the functions associated to these messages
    • Added the Agent's launch system in apps.py
  • Implementation of the VOEventListener Thread
    • Created a "class VOEventListener(Thread):" in agent.py
    • Overriding the run method
  • 19/05/2016 : deleted the agent system (deleted from the applications below too)
  • Added some VOEvents in events_to_send
  • Created 2 celery tasks in tesks.py : alert_listener and change_obs_strategy
  • Added alert_listener launch at server start
  • Implemented alert_listener : it waits for an event file to be created in events_received, then read it and creates a request
  • Added a simulator system ( see section Tests below )
  • Added the strategy_change page
  • Removed launching at start : it is now started by the monitoring task

Tests

  • Simulator :
    • You first need to set settings.SIMULATION to True (in pyros/settings.py)
    • Start celery workers (scripts/start_celery_workers.sh)
    • Start server (manage.py runserver)
    • Go to localhost:8000/alert_manager/simulation, and press the button !
    • This will read into the simulation_sequences file.
  • Test AlertListenerTestsCelery.test_alert_reception :
    • Tests if the alert_listener is working well.
    • Be careful : this will act on the production database, and will start the entire alert workflow
    • To start it : script/celery_test.sh alert_manager
  • Tests TestStrategyChange.* : * Test the changing of the observation strategy with errors and different parameters

Notes

  • The request created by alert_listener is hard-coded

analyzer

Purpose

  • Apply calibration files to the images taken by the observation manager
  • Analyze the images produced by the calibrations

Evolution

  • Creating application
    • See scheduler documentation above
  • Implemented the skeleton of the module in the task Analysis (call a function for calibrations then a function for analyses)

Notes

majordome

Purpose

  • Handles the execution of the sequences in the right order during the night
  • Handles the system start & stop
  • Handles the changes of observation conditions
  • Stops the celery tasks when needed

Evolution

  • Creating application
    • See scheduler documentation above
    • Created all the tasks in tasks.py
    • Created the TaskManager to stop the sequence & plan executions when an alert / routine is received
    • Implemented execute_sequence (not finished)

Notes

monitoring

Purpose

  • It is the only tasks launched at the system's start
  • Make all the starting actions : instruments configuration, connection to the PLC, updating software versions, determine night start/end & start a scheduling
  • Starts the alert_listener
  • Infinite loop to regularly check the instruments and PLC status (and send them to the IC)
  • Handles night start & end

Evolution

  • Creating application
    • See scheduler documentation above
  • System to launch the task at the system's start
  • Implement skeleton of task
  • Implement starting actions (with some empty functions)
  • Main loop to check PLC, instruments and night start / end (not finished)

Notes

observation_manager

Purpose

  • Give orders the instruments to execute the plans
  • Take the calibration images and computes the 'super' calibration images

Evolution

  • Creating application
    • See scheduler documentation above
  • Created the tasks : execute_plan_(nir/vis) & create_calibrations
  • Implemented the execute_plan tasks with some incomplete functions

Notes

routine_manager

Purpose

  • Enables astronomers to create routines via the website
  • Import / export of routines (XML)
  • Display observation results for routines

Evolution

  • Creating application
    • See scheduler documentation above
  • Created forms online for routine creation
  • Added import / export
  • Added routine submit / unsubmit

Tests

  • Import fake file
  • Import invalid file
  • Import good file
  • Export incomplete
  • Export valid

Notes

user_manager

Purpose

  • Handle user create / delete / login / logout
  • Allow to see user profile and make password recovery

Evolution

  • Creating application
    • See scheduler documentation above
  • Taking and adapting templates from Alexandru project
  • User creation
    • Form in admin.py
    • Functions to verify if email doesn't exist and if passwords match
    • Added check to pass login page if user is already autheticated
    • Restricted access to all pyros views to authenticated users (except from user creation and login) with @login_required

Tests

  • Creation
  • Login
  • wrong email
  • wrong password
  • logout

Notes

common

Purpose

  • Regroups common data and functions shared by applications
  • Contains all the database Models
  • Basic tests in tests.py
  • Backoffice configuration in admin.py

Content

  • RequestBuilder - class to build and save in DB the requests
    • You can start a request, then add sequences, albums and plans, given their parent class (ex: you need to give a sequence id to create an album)
    • Each function returns the ID of the element it created (to give it to the children classes)
    • To save a request, you need to use validate_request
  • Device and children
    • Implementation of devices communication, messages and initialization
    • All classes inherit from the Device class
    • All classes re-implement the list of messages

Tests

  • Test the creation of a full request by the RequestBuilder
  • Basic tests for models

Notes

  • Will probably contain the DB models at last ...