Marspedia - User contributions [en]

MDRS:CrewReportsLog

2020-12-31T21:31:14Z

B00klegger: First draft

MDRS crew reports processing pipeline

The MDRS email archive consists of all official emails sent between Mission Control, habitat crews and other Mars Society staff over the past twenty years.

There are several email archive files to process; each one comprises multiple years of reports. Each file is multi-gigabyte and contains all of the emails as a single flat file.

The first archive comprises the years 2011-2019; a total number of emails is about 18000.

The file is a flat text file formatted as an email export. Each email is comprised of a lengthy header followed by text and HTML versions of the email along with any attachments formatted as base64 text.

<nowiki>https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html</nowiki>

The link details the structure of emails in a raw format. After the header the different sections of the email are split by a 'boundary' field. Each section have its own subset of fields which define the contents of that section.

A overall boundary field is defined in the primary header. This boundary is a kind of 'wrapper' boundary; it defines the start and end of an email at a minimum. Within the body of the email the different sections reference the boundary to demark different sections of an email.

The emails typically have multiple sections:

* A 'text/plain' content type, the flat text of an email
* A 'text/html' content type, the html version of the email
* A binary content type, 'application/zip' or 'image/jpg' for example, a base64 text representation of a file attachment

Additionally there are additional internal boundaries which are defined in their own internal sub-header and demarks subsections. A wrapper inside of a wrapper.

Email is an open standard of mutually incompatible implementations.

The process has been broken down into several stages:

* Scan the master email file and split into individual emails
* Process each individual email and split into separate sections (text, html, attachments)
* Process each email header and collect data about each email:
** To
** From
** Date
** Subject
** Boundary fields
** Thread ID

The next stage will require parsing the subject and email body sections looking for keys words. As part of our work with MDRS researchers we will need to locate and mark up the emails with this data:

* Type of Report (commander, science, operations, etc)
* Person who submitted
* Position of person who submitted
* Crew #
* Field Season #
* Date submitted
* Sol # of crew rotation (if available)
* Text of report
* Any photo attachments of report
* Any other file attachments

MDRS:CrewReportsLog

2020-11-30T01:53:08Z

B00klegger: Process the crew reports from MDRS and tag with metadata

MDRS crew reports processing pipeline

MDRS:HAL Development Log

2018-10-13T16:28:40Z

B00klegger: Updated the name of the link

[[File:HAL.png|none|thumb|238x238px|HAL]]

==Development Log of the MDRS Habitat Activity Lexica (HAL)==
Developers: James Burk, Ryan Watson, Matt Hopson (MarsChat)

==Operating System==

Ryan Watson:
I have a beta Raspberry Pi image that I've posted up on S3 and I want to invite you to download, write to your SD card and give it a try. It has a very sparse selection of apps:

*Chromium (Can be changed to Firefox, defaults to Chromium so left it alone for now)
*VLC
*Libreoffice (Default in the base image)
*OMX

==Users==

Default pi user still in effect as admin. Likely disable but not delete, due to potential future needs of having that standard user.

There are two baked in users, an administrative user for possible future tasks and a workstation user:
- U: `admin` / PW: `get_to_mars`
- U: `mdrs` / PW: `get_to_mars`

==Current Issue List==

#need to fix sudoers, still is expecting the default `pi` user
##command to disable: sudo passwd --lock pi
#Add Mars related wallpaper by default
##[http://mdrs.marssociety.org/wp-content/uploads/2017/10/MDRS-2017.jpg MDRS] or [http://mdrs.marssociety.org/wp-content/uploads/2017/10/URC.jpg URC]
#Fixup timezone, initial / ongoing time sync via ntp. Currently defaults to UTC
#Password is currently set during the image build / provision. Can / should be more dynamic
#Dual displays with HDMI output as primary
##add "display_default_lcd=0" to /boot/config.txt

==Install walk-thru==
The end goal is to write the boot image file to an SD card, configured to boot a Raspberry Pi. There are several possible workflows possible using OSX, Linux or Windows. I will detail a Windows based workflow below and add others as necessary.

===Windows 10===
1) Download artifact from S3

The zip file artifact is currently located at [https://s3.amazonaws.com/watson-builds/image_2018-10-12-razpi-modz-v.5.2.zip https://s3.amazonaws.com/watson-builds/image_2018-10-03-razpi-modz-v.5.2.zip] and downloadable via browser or other tool.

2) Install image writer utility

I found via the Raspberry Pi forums a utility called etcher written and hosted by resin.io, an IoT company. The utility is straightforward and easy to use. There is no need to extract the img file from the downloaded zip file. This utility can accept the compressed file and write to an SD card.

Download: https://etcher.io/

3) Insert SD card

Before starting Etcher it is helpful to insert the SD card then start the application.

4) When the application is open navigate to the download location and select the still compressed .zip file

5) The SD card should already be selected, if not choose the correct target SD card drive.

6) Flash!

Etcher will begin to write the image to the card. In testing this typically takes about 15 minutes. During the process Windows will notify that the drive has been ejected, or that the card needs to be formatted before use etc. Cancel or dismiss these messages and let Etcher continue. After the write is complete it will verify the write and then software eject the SD card (unmount it from the computer). You can now remove the SD card and place into the RaspberryPi.

MDRS:HAL Development Log

2018-10-13T16:27:18Z

B00klegger: /* Development Log of the MDRS Habitat Activity Lexica (HAL) */

[[File:HAL.png|none|thumb|238x238px|HAL]]

==Development Log of the MDRS Habitat Activity Lexica (HAL)==
Developers: James Burk, Ryan Watson, Matt Hopson (MarsChat)

==Operating System==

Ryan Watson:
I have a beta Raspberry Pi image that I've posted up on S3 and I want to invite you to download, write to your SD card and give it a try. It has a very sparse selection of apps:

*Chromium (Can be changed to Firefox, defaults to Chromium so left it alone for now)
*VLC
*Libreoffice (Default in the base image)
*OMX

==Users==

Default pi user still in effect as admin. Likely disable but not delete, due to potential future needs of having that standard user.

There are two baked in users, an administrative user for possible future tasks and a workstation user:
- U: `admin` / PW: `get_to_mars`
- U: `mdrs` / PW: `get_to_mars`

==Current Issue List==

#need to fix sudoers, still is expecting the default `pi` user
##command to disable: sudo passwd --lock pi
#Add Mars related wallpaper by default
##[http://mdrs.marssociety.org/wp-content/uploads/2017/10/MDRS-2017.jpg MDRS] or [http://mdrs.marssociety.org/wp-content/uploads/2017/10/URC.jpg URC]
#Fixup timezone, initial / ongoing time sync via ntp. Currently defaults to UTC
#Password is currently set during the image build / provision. Can / should be more dynamic
#Dual displays with HDMI output as primary
##add "display_default_lcd=0" to /boot/config.txt

==Install walk-thru==
The end goal is to write the boot image file to an SD card, configured to boot a Raspberry Pi. There are several possible workflows possible using OSX, Linux or Windows. I will detail a Windows based workflow below and add others as necessary.

===Windows 10===
1) Download artifact from S3

The zip file artifact is currently located at [https://s3.amazonaws.com/watson-builds/image_2018-10-12-razpi-modz-v.5.2.zip https://s3.amazonaws.com/watson-builds/image_2018-10-03-razpi-modz-v.2.zip] and downloadable via browser or other tool.

2) Install image writer utility

I found via the Raspberry Pi forums a utility called etcher written and hosted by resin.io, an IoT company. The utility is straightforward and easy to use. There is no need to extract the img file from the downloaded zip file. This utility can accept the compressed file and write to an SD card.

Download: https://etcher.io/

3) Insert SD card

Before starting Etcher it is helpful to insert the SD card then start the application.

4) When the application is open navigate to the download location and select the still compressed .zip file

5) The SD card should already be selected, if not choose the correct target SD card drive.

6) Flash!

Etcher will begin to write the image to the card. In testing this typically takes about 15 minutes. During the process Windows will notify that the drive has been ejected, or that the card needs to be formatted before use etc. Cancel or dismiss these messages and let Etcher continue. After the write is complete it will verify the write and then software eject the SD card (unmount it from the computer). You can now remove the SD card and place into the RaspberryPi.

MDRS:HAL Development Log

2018-10-06T20:19:13Z

B00klegger: /* Development Log of the MDRS Habitat Activity Lexica (HAL) */

[[File:HAL.png|none|thumb|238x238px|HAL]]

== Development Log of the MDRS Habitat Activity Lexica (HAL) ==
Developers: James Burk, Ryan Watson, Matt Hopson (MarsChat)

==Operating System==

Ryan Watson:
I have a beta Raspberry Pi image that I've posted up on S3 and I want to invite you to download, write to your SD card and give it a try. It has a very sparse selection of apps:

*Chromium (Can be changed to Firefox, defaults to Chromium so left it alone for now)
*VLC
*Libreoffice (Default in the base image)
*OMX

==Users==

Default pi user still in effect as admin. Likely disable but not delete, due to potential future needs of having that standard user.

There are two baked in users, an administrative user for possible future tasks and a workstation user:
- U: `admin` / PW: `get_to_mars`
- U: `mdrs` / PW: `get_to_mars`

==Current Issue List==

#need to fix sudoers, still is expecting the default `pi` user
#Add Mars related wallpaper by default
#Fixup timezone, initial / ongoing time sync via ntp. Currently defaults to UTC
#Password is currently set during the image build / provision. Can / should be more dynamic

== Install walk-thru ==
The end goal is to write the boot image file to an SD card, configured to boot a Raspberry Pi. There are several possible workflows possible using OSX, Linux or Windows. I will detail a Windows based workflow below and add others as necessary.

=== Windows 10 ===
1) Download artifact from S3

The zip file artifact is currently located at https://s3.amazonaws.com/watson-builds/image_2018-10-03-razpi-modz-v.2.zip and downloadable via browser or other tool.

2) Install image writer utility

I found via the Raspberry Pi forums a utility called etcher written and hosted by resin.io, an IoT company. The utility is straightforward and easy to use. There is no need to extract the img file from the downloaded zip file. This utility can accept the compressed file and write to an SD card.

Download: https://etcher.io/

3) Insert SD card

Before starting Etcher it is helpful to insert the SD card then start the application.

4) When the application is open navigate to the download location and select the still compressed .zip file

5) The SD card should already be selected, if not choose the correct target SD card drive.

6) Flash!

Etcher will begin to write the image to the card. In testing this typically takes about 15 minutes. During the process Windows will notify that the drive has been ejected, or that the card needs to be formatted before use etc. Cancel or dismiss these messages and let Etcher continue. After the write is complete it will verify the write and then software eject the SD card (unmount it from the computer). You can now remove the SD card and place into the RaspberryPi.