Information


NCMAS 2017 Computational Resources

The National Computational Merit Allocation Scheme (NCMAS) is the premier, competitive program for allocation of high-performance computing resources for scientific research in Australia. NCMAS will allocate HPC resources from the following participating facilities for the 2017 calendar year.

You can view the resouces allocated here.

System Computing Time (KSU)
NCI: Raijin (Fujitsu) and Raiju (Dell) 100,000
Pawsey Centre: Magnus (Cray) 100,000
Monash: MASSIVE (SF Visualisation) 1,600
UQ: FlashLite (Xenon) 1,930

Note that all references to computational resources will be in units of kilo-service-units; 1 KSU = 1000 service units (SU). Service units, nominal cpu-hours, are scaled to be equivalent across all participating HPC facilities.

National Computational Infrastructure (NCI)
Facility overview

NCI is Australia's national research computing service. Home to Raijin (Fujitsu), the nation's most highly integrated and highest performance supercomputer, NCI provides innovative, world-class services to Australian researchers. NCI operates a formal collaboration between Australia's national university - ANU; the national research agency - the Commonwealth Scientific and Industrial Research Organisation; the national meteorological agency - the Australian Bureau of Meteorology; and the national geosciences agency - Geoscience Australia. Since 2007, NCI's partnership has expanded to include a further 22 Australian universities. NCI's infrastructure was established through Commonwealth Government funding. It includes a 1.2 petaflop supercomputer, that has now been expanded to host 120 nVidia K80 GPUs, and from January 2017 will receive a boost in performance and capacity with the addition of the latest generation Broadwell CPUs supporting the 512 AVX instruction set. Additional resources at NCI that may be leveraged include a 3,600-core compute cloud, data storage in excess of 14 petabytes and a purpose-built data centre.

Raiju (Dell) is a new computing capability which will be available to users from January 2017. Raiju will be comparable to Raijin in terms of hardware specifications and will have access to Raijin’s filesystem and software stack.  Raiju is being provisioned to accommodate proposals requesting 100-249 KSU/year due to strong demand for this level of project resourcing. Requests larger than 250 KSU/year will be accommodated on Raijin. 

NCMAS computing resources 100 M core hours on Raijin and Raiju. 
NCMAS storage resources 750 TB available to NCMAS scheme. Commitee to allocate according to project requirement.
Software NCI maintains more than 170 application software packages for use on its systems. The NCI application software catalogue is available online at http://nci.org.au/user-support/getting-help/application-software/.
User support NCI operates an expert Service Desk for users during normal business hours, Mon-Fri between 9:00 am and 5:00 pm AEST. NCI Academic Consultants can provide assistance with user and project registration and operational issues, and can provide advice on code development and performance, and the use of scientific software in HPC environments. The User Services team aims to resolve help requests within four working hours.
Pawsey Supercomputing Centre
Facility overview

The Pawsey Supercomputing Centre (Pawsey) is one of two NCRIS funded national High Performance Computing centres.  Pawsey is a joint venture of the CSIRO, Curtin University, Edith Cowan University, Murdoch University and The University of Western Australia, and it provides services to all Australian computational researchers through schemes including NCMAS.

Pawsey has an array of scientific computing instruments available for researchers, including a machine for operational radio astronomy (Cray XC-30 Galaxy), machines for data analysis and processing, and Australia's most powerful research supercomputer the Cray XC-40 system called Magnus.

NCMAS applicants can request time on the Cray XC-40 system Magnus, which has 35,572 Intel E5-2690v3 cores.  The processors are arranged in nodes of 24 cores, with 64 gigabytes of memory per node, and there is a scratch file system capable of a sustained I/O bandwidth of approximately 70 gigabytes per second.  Magnus has a very high performance Cray network with a dragonfly topology, and Cray's tuned Linux environment and software stack.

Magnus is particularly suited for compute problems (i.e. application codes and datasets) that have high network bandwidth requirements, and/or scalable problems that would benefit from the Cray interconnect.

NCMAS computing resources 100 M core hours on Magnus.
NCMAS storage resources Storage is available on a group filesystem which currently has a total of 750 terabytes available for all Pawsey projects.  Each project is allocated 1 terabyte by default, and up to 10 terabytes can normally be accommodated.  Requests for more than 10 terabytes of storage need to be justified with reference to the files and data to be stored, a capacity plan over the project duration, and any compression techniques and data lifetime purge policies employed.
Software A list of currently installed software is available at https://support.pawsey.org.au/documentation/display/US/Software+Map however this list is under review and researchers should clearly articulate their actual requirements in their NCMAS submissions.
User support The Pawsey Help Desk is available by email between 9am and 5pm AWST daily except for Western Australian public holidays and the extended Christmas closure period.  In addition to the general helpdesk, additional application and optimisation support is available, as well as training courses in the use of the Pawsey systems.
Additional notes Applications for time on Magnus must demonstrate the ability to make effective use of the architecture by running scalable applications and research problems.
NCI Specialised Facility in Imaging and Visualisation (MASSIVE)
Facility overview The Multi-modal Australian ScienceS Imaging and Visualisation Environment (MASSIVE) is the Australian specialised high performance computing (HPC) facility for imaging and visualisation. MASSIVE underpins a range of advanced imaging modalities, including synchrotron X-ray and infrared imaging, functional and structural magnetic resonance imaging (MRI), X-ray computer tomography (CT), electron microscopy and optical microscopy. The facility is partnered with the Centre of Excellence in Advanced Molecular Imaging and the Center of Excellence in Integrative Brain Function, and provides a wide range of tools and services to molecular science and neuroscience researchers. MASSIVE provides easy access through a dedicated remote desktop environment and supports a wide range of users new to HPC. The facility runs two HPC systems, M1 and M2, both with fast parallel file systems and NVIDIA GPUs. Further technical information: https://www.massive.org.au/high-performance-computing/resources
NCMAS computing resources 1.6 M core hours on M1 and M2 machine.
NCMAS storage resources A parallel file system providing a combined speed of 3GB/s write. Projects are provided with 1TB space by default, which can be increased for specific use cases.
Software A list of software available on MASSIVE is available at: https://www.massive.org.au/userguide/software-instructions/installed-software.
User support
  • New HPC users: strong support for new HPC users through simple remote desktop access. MASSIVE runs a helpdesk for assistance, and provides consultation services for users and instrument facilities. Most user assistance tickets are answered within hours.
  • Instrument integration: strong capability in developing near-realtime analysis workflows for instruments and experiments;
  • Data processing: strong experience in data processing, in-particular large cohort data studies, volumetric data and imaging data;
  • Visualisation: support for a range of visualisation tools accessible through the MASSIVE Desktop, and large-scale multi-node visualisation.
  • Molecular science: processing workflows for structural biology data, including MX and CryoEM. 
  • Brain research: supports a large selection of brain (both psychological and fundamental) science projects, with a particular emphasis on MRI data processing. In 2016 onward, MASSIVE will host an Australian mirror of the Human Connectome Project.
University of Queensland Research Computing Centre (Flashlite)
Facility overview FlashLite is a research computer that has been designed explicitly for Australian research to conduct data intensive science and innovation. FlashLite is optimised for data intensive computation and has 1632 cores, 34.8 TB of RAM, 326.4 TB of NVMe storage and 65.28 TFLOP/s (Rpeak) Compute nodes in FlashLite can be flexibly aggregated together into larger "supernodes" using ScaleMP’s vSMP software. Given the network topology, a supernode of 22 physical compute nodes with 528 cores and 11TB of shared memory is a likely practical limit but larger supernodes are also possible.
NCMAS computing resources Total core hours for NCMAS on FlashLite is 1.93 MSU.
NCMAS storage resources There is a limited amount of network attached GPFS filesystem storage available within FlashLite. FlashLite is co-located with the QRIScloud research data storage. It is envisaged that NCMAS projects would make an application for a research data collection based at QRIScloud.
Software Details of software available on FlashLite will be published on RCC website and elsewhere as the information becomes available.
User support Only basic user support will be available for applications and external groups using FlashLite.
Additional notes The workloads for FlashLite must be demonstrated to be data intensive or very large memory in character.

Schedule

Key dates for the NCMAS 2017 application round are listed in the table below.

Date Event
5/09/2016 (Mon) Applications open
21/10/2016 (Fri) Applications close 8pm (AEDT) and 5pm (AWST)
28/10/2016 (Wed) Technical assessments completed
23/11/2016 (Wed) Merit assessments completed
28/11/2016 - 29/11/2016 (Mon-Tue) Allocation Committee meeting
5/12/2016 (Mon) Applicants and facilities notified of outcomes

Eligibility

CHIEF INVESTIGATORS

In keeping with guidelines for access to Commonwealth-funded research infrastructure, and relevant Australian Government legislation, Chief Investigators of NCMAS projects must be:

  • Australia-based researchers;
  • hold a substantive position at a higher-education institution or research institute (which are eligible to receive funds from the Australian Research Council or the National Health and Medical Research Council);
  • or an Australian publicly-funded research agency (e.g. CSIRO, BoM, Geoscience Australia, ANSTO, etc.)

A researcher undertaking a higher degree by research is not eligible to be a Chief Investigator on a NCMAS proposal.

A individual may be Lead Chief Investigator on one NCMAS 2017 application only.

EARLY CAREER RESEARCHERS AND SPECIAL ALLOCATIONS

A limited number of special allocations will be made available to Early Career Researchers (ECR), new appointments, or others who may otherwise not be competitive in the NCMAS. These allocations are to enable such researchers to establish a presence and track record on the national facilities. ECR applications will be considered by the NCMAS merit allocation committee independently of the main NCMAS application pool. ECR allocations will be a fixed award of 250 KSU per year. An applicant may apply for ECR or special consideration for a maximum of three consecutive years. Proposals for special allocations should be a maximum of 1000 words, or approximately two pages in length. These allocations are highly competitive, therefore recipients would be expected to hold or be competitive for a junior personal research fellowship, such as an ARC DECRA or NHMRC CDA. Researchers are eligible to apply for an Early Career Researcher award if they have been awarded a PhD within the last five years or, together with periods of significant career interruption, have been awarded a PhD within nine years of the closing time for submission of applications. 

CONDITIONS OF USE

All NCMAS applicants (Lead Chief Investigators) will be required to sign and return a certification for conditions of use and compliance with recent Commonwealth legislation, in particular, the Autonomous Sanctions Act (2011, Cth) and the Defence Trade Controls Act (2012, Cth), which impose additional requirements for supporting documentation and certification of project researchers in some circumstances. Applicants should review the current conditions of use available on facility websites (follow links to the individual facility web sites.)

EMAIL ADDRESSES

To ensure compliance with relevant Australian Government legislation, all researchers named in NCMAS applications must register and use an official institutional email address for all correspondence. Chief Investigators and researchers who currently use a non-institutional email address (for example, @gmail.com or @yahoo.com) must register an official institutional email address before their NCMAS project application can be accepted and processed. Please note that it is the responsibility of the Lead Chief Investigator to ensure that all project staff register and use official email addresses.


Proposal Guidelines

ASSESSMENT AND ALLOCATION

The assessment of proposals will be based on a combination of technical and scientific merit (see selection criteria section) based on material provided in the proposal. All applicants are expected to provide a detailed justification of the resources requested, and are expected to demonstrate the capacity to utilise the requested HPC resources effectively. All requests must be proportional to the scientific merit of the proposal. The NCMAS Committee reserves the right to allocate all or part of the resources available, and all or part of any specific request.

Minimum request

Each facility will set a minimum threshold (cpu-hours per year) for resource requests. The minimum request value for each facility is listed in the application form. Applicants requiring less than the NCMAS minimum allocation are encouraged to approach their home institution, regional or state-based HPC consortia (i.e. Intersect, QCIF, eRSA, TPAC), or partner schemes on the national facilities held by their institution with their resource request. 

Minimum allocation

Each facility has set a minimum value for NCMAS allocations, as listed in the table below. Proposals assessed as not requiring the minimum allocation at a selected facility will not be awarded NCMAS resources at that facility.

System Minimum Allocation (KSU)
NCI: Raijin (Fujitsu) 250
NCI: Raiju (Dell) 100
Pawsey Centre: Magnus (Cray) 250
Monash: MASSIVE (SF Visualisation) 50
UQ: FlashLite (Xenon) 20
Maximum request

Resource requests for NCMAS are not subject to a maximum limit (cap.) This is to allow researchers to prepare a single proposal that can be considered by multiple allocation schemes, and which reflects their actual need for HPC resources. A request for more than 4000 KSU per year on any facility would normally be associated with teams of experienced researchers who can clearly demonstrate a track record of efficient and productive use of HPC resources. Note that any proposal deemed not to have fully justified the resources requested will be rejected.

Maximum allocation

There is no formal maximum allocation for NCMAS. Allocations will reflect the relative merit of proposals, and the demand for resources from the facilities. As a general rule, allocations of more than 5,000 KSU per year are highly unlikely and will only be made to exceptional projects involving researchers with an outstanding record of success. The maximum allocation on Raijin for 2017, for example, would be expected to be approximately 10 per cent of the available compute resource, or 10,000 KSU/year. 

Quantised allocations

Computing resources will be allocated in specific increments (quanta) as indicated on the NCMAS application form. Allocations are quantised to simplify the work of the allocation committee.

PROPOSAL LENGTH

The length and rigour of a proposal should be proportional to the requested allocation. Guidelines for the length of proposals are listed in the table below. Note that 500 words is approximately one page of single-spaced, 12-point type, with 2 cm margins.

Resource Request (KSU) Proposal Length (words)
Greater than 4000 3000
1000 - 4000 2000
100 -1000 1000

FORMAT

Use plain English and comply strictly with the proposal format and submission requirements. Use Australian English spelling.

All pages of additional text (uploaded in PDF form) must be as follows:

  • Black type, or occasional coloured type for highlighting purposes
  • Single column
  • White A4 size paper with at least 0.5 cm margin on each side, top and bottom
  • Text must be size 12 point Times New Roman or an equivalent size before converting to PDF format and must be legible to assessors. Otherwise, a highly legible font type must be used: Arial, Courier, Palatine and Helvetica subject to them being an equivalent size to 12 point Times New Roman. Variants such as mathematical typesetting languages may also be used.
  • References only can be in 10 point Times New Roman or equivalent
  • Adhere strictly to page limits designated for each part of the proposal.
  • Applicants should note that colour graphs, colour photographs, detailed graphics, and grey scale objects may be reproduced in black and white.
  • Additional text uploaded as PDF may appear slightly reduced in size due to NCMAS formatting of attachments. Additional text uploaded in PDF form should be directly generated rather than scanned to maximise the quality of reproduction.
  • The NCMAS Secretariat reserves the right to seek an original electronic copy of the Proposal to determine that the text meets these requirements.

Assessments

ASSESSMENT CRITERIA

Assessment Area

Project quality and innovation

  • Significance of the research
  • Originality and innovative nature of the computational framework
  • Advancement of knowledge through the goals of the proposed research
  • Potential for the research to contribute to Australian Science and Research Priorities

Investigators

  • Research record and performance relative to opportunity (publications, research funding, recognition and esteem metrics)

Feasibility

  • Adequacy of the time commitment of investigators to undertake the research and utilise the resources successfully
  • Capacity to realise the goals of the project within the resources request
  • Appropriate track record in the use of high-performance computing systems, relative to the scale of the resources requested
  • Suitability of the system to support the research, and an appropriate and efficient use of the system

Benefit and impact

  • Ability of the project to generate impactful outcomes and produce innovative economic, environmental and social benefits to Australia and the international community

Acknowledgements

A condition of accepting an allocation of resources through the NCMAS is that applicants acknowledge both the scheme and the specific facility used in all publications and presentations of the associated work. For example:

This work was supported by computational resources provided by the Australian Government through <facility_name> under the National Computational Merit Allocation Scheme.


Appeal Process

All decisions of the NCMAS Allocation Committee are final. Appeals will be considered only against adminstrative or procedural issues and not against decisions of the Committee or against assessor ratings and comments - consistent with the practices of the Australian Research Council. Administrative appeals must be submitted by the project Lead Chief Investigator using the form available on the NCMAS web site. Appeals must be submitted within 28 working days of the announcement date for NCMAS outcomes. All appeals will be considered by the NCMAS Allocation Committee chair, and processed within 28 working days of the adminstrative appeal deadline. 

Administrative appeals must be submitted online at https://ncmas.nci.org.au/2017/appeal.

Functions of the appeal process are to:

  • determine whether there has been any error in the administrative process relating to the selection process;
  • determine whether any such errors led to a defect in decision making by the NCMAS secretariat and/or committee;
  • determine whether to uphold or dismiss an appeal;
  • provide advice to the NCMAS committee and secretariat in relation to how NCMAS administrative processes could be modified or improved.

Contact

If you have questions about the application process please contact the NCMAS Secretariat at ncmas@nci.org.au.

Best regards,
NCMAS Secretariat