Option 1: Using the Redivis Transforms & Notebooks
A Versatile Integrated Option
Redivis is the platform that hosts the PHS Data Portal. In addition to storing datasets and managing the data access application process, the platform also has robust data wrangling and analytic tools. In Redivis Workflows, researchers can use these tools for their computational needs without switching between platforms. It is free for most use-cases, but can also be customized for more intensive computational needs.
Recommended for: Most users, especially for collaborations with individuals from outside institutions.
Optimal Use
- R (no RStudio)
- Python
- SQL
- Stata (no license needed, but interface is slightly different from the desktop application)
- SAS (but interface is somewhat different from the desktop application)
- Is Cloud-based.
Costs
Free for most uses. You can pre-buy extra computational resources for the notebooks if needed. Costs for extra resources are transparent and predictable and are roughly equivalent to GCP costs. You can also pre-set limits on resource use to control costs.
Benefits
- Low/No Cost: In most cases (additional details in 'Cost' section above).
- Easy access: No need to transfer any data between environments; it's all already on the platform. Additionally, if datasets are updated, that update will automatically be made available to you without the need to move any data.
- Stability: Because Redivis is based in Google Cloud, downtimes are extremely rare.
- Power: If you need extra GPUs or wish to perform computationally-intensive methods, the parameters of your notebooks can easily be adjusted to accommodate your needs.
- Flexibility: You can use a point-and-click interface to process data, or code in your language of choice.
- Enhanced collaboration: Multiple people can work on a project and view each other's edits simultaneously. It is also easy to share code without risking data security.
- Visualization and summary features: Distributions and statistical summaries are automatically generated for each variable in each transform in the data cleaning and analysis process.
- Training capabilities: New users can get practice working with data, implementing methodologies using the point-and-click interface (for which the corresponding SQL code is automatically generated and can be viewed) or by writing code, all supported with available tutorials and practical documentation.
Drawbacks
- Coding in the notebooks can feel a bit different from coding in desktop applications, and some users do feel it can take some getting used to.
- Code must be manually downloaded if one wishes to put it on GitLab.
- May not be ideal for Stata or SAS. Both languages are supported in the Notebooks, but the Jupyter-style environment may be less familiar to those accustomed to the desktop applications.
Additional Redivis Resources
Overview of Transforms: https://docs.redivis.com/reference/workflows/transforms (on the left side of the page, you can see all of the possible function ‘steps’, and if you select one, you’ll be shown the different parameter options, and details about how they work)
Transform Instructions: https://docs.redivis.com/guides/analyze-data-in-a-workflow/reshape-data-in-transforms
Overview of Notebooks: https://docs.redivis.com/reference/workflows/notebooks/notebook-concepts (and you can navigate to code samples in various languages on the left side of the page)
Notebook Instructions: https://docs.redivis.com/guides/analyze-data-in-a-workflow/work-with-data-in-notebooks
Notebook Demonstration: Video
Data Analysis and machine learning workflows on Redivis Demonstration: Video
Redivis Office Hours: https://calendly.com/phsdatacore/redivis
Redivis Slack Channel: #redivis