I currently work for the premier distributor of historical financial data. My role primarily entails leadership and data architecture. I'm responsible for making technology and business decisions that improve business performance, increases profits and leads to future opportunites. I get to work with an amazing group of talented people, helping guide them in the direction needed to achieve the goals that we're set out to accomplish for the wellbeing of the company. At the same time, I ensure that my team and colleagues are able to achieve their own career goals as well. I also work with sales, providing data that allows them to identify clients and the products that are most likely to appeal to them. And the executive team to logically clarify where we are, where we're going and what we're capable of accomplishing. Additionally, I get to research and implement solutions that allows us to scale the business and improve upon what what we do and how we do it.
What data technologies am I working with?
SQL Server, MariaDB, Azure SQL, Azure Database for MariaDB - The database engines that we use to store and return OLAP and OLTP data.
Python - Used to scrape data, chart data, graph data, analyze data, automate database schema/dictionary documentation and for various application development.
Power BI - Used to quickly and effectively build and deploy reports.
PowerShell - Used to automatically upload data and file backups/archives to Azure, validate archives, work with Active Directory. Basically for DevOps.
Visual Studio - One of the best IDEs for application development and code managment.
Visual Studio Code - One of the best code editors for scripting languages. Great for web development, Python, Java, TypeScript, etc.
C# - For legacy data load processes. Also used in some SSIS costomizations, such as data conversions.
SSMS - For Microsoft SQL Server database management.
SSIS, Azure Data Factory - For Microsoft-based ETLs and ELTs for data pipelines.
SSAS - Used to perform multi-dimensional data analysis.
MySQL Workbench - My primary IDE for MariaDB database management.
HeidiSQL - My secondary IDE for MariaDB database management.
Docker - Used to quickly and effortlessly containerize software configurations, replicate container images, deploy containers, and build distributed systems.
Apache Spark - Used to improve the performance of working with large data sets. Also works great for data/delta lakes.
Apache Airflow - Used for non-Microsoft data pipelines and process flows. Allows us to orchestrate processes (ETL, ELT, data downloads, data uploads, process execution, etc.), allowing us to add flows, dependencies, etc.
TFS, Git - Used to control change tracking of scripts, solutions, projects, documents, etc.
Linux - An effective OS that is lighter than Windows and MacOS. Linux has come a long way in the last decade.
Apache Kafka - Used for event-based data and file streaming.
Note that I left out a number of technolgies that are fairly innate to most technologists such as Outlook, MS Office, Trio Office, OpenOffice, Google, Excel, web browsers, etc.