Viewing Logs

Each data processing session that you run on a cluster has logs. These logs include errora details in the case of a session failing.

Viewing Session Logs

The standard output and standard error from the session running on a cluster is redirected to a log file stored on disk on the cluster. If the print_logs parameter is set to true when the session was started, the logs will get streamed and printed out on the client side as code gets executed on the cluster. When the session is ended, if the store_logs_in_s3 parameter was set to true when the session was started, this log file will be persisted in cloud storage in your AWS account. You can either view this log file from the Banyan dashboard or download it to your local machine for viewing using the client library.

To view the log file from the Banyan dashboard, navigate to the Sessions tab, search for the session based on its session ID or session name, and click on the button to view logs.

To download the log file to your local machine, use download_session_logs().

JuliaPython

download_session_logs(
    session_id="2021-06-10-2240237175e6921d0c69f8a93aaa49ffd30287",
    cluster_name="weekly-data-playground"
)

download_session_logs(
    session_id="2021-06-10-2240237175e6921d0c69f8a93aaa49ffd30287",
    cluster_name="weekly-data-playground"
)

Viewing Cluster Logs

You should usually not need to view any logs for your cluster. However, if cluster creation fails and the information provided by get_cluster_status() is not sufficient, you may want to view more detailed logs. You can find the logs at the following URL: https://<AWS_REGION>.console.aws.amazon.com/cloudwatch/home?region=<AWS_REGION>#logsV2:log-groups/log-group/$252Faws$252Fparallelcluster$252F<CLUSTER_NAME>