Danielle's Blog

I Rewrote My Step Function as a Durable Function

Danielle Heberling — Sun, 01 Mar 2026 12:12:03 GMT

Ever since AWS announced <a href="https://docs.aws.amazon.com/lambda/latest/dg/durable-functions.html" target="_blank" rel="noopener noreferrer">Lambda durable functions</a> at re:Invent 2025, I've been wanting to try them on a real project. Not a cookie cutter tutorial, but something where I could actually compare the two approaches. I already had the perfect candidate: my <a href="https://github.com/deeheber/weather-site" target="_blank" rel="noopener noreferrer">weather site</a>.

Quick Background

If you haven't read my original blog post, here's the short version. I built <a href="https://isitsnowinginhillsboro.com/" target="_blank" rel="noopener noreferrer">isitsnowinginhillsboro.com</a> because the existing Portland snow site wasn't accurate for my area. It's a serverless workflow that checks the <a href="https://openweathermap.org/api" target="_blank" rel="noopener noreferrer">OpenWeatherMap API</a> every 10 minutes and updates a static S3 site with a YES or NO answer. The original uses a Step Function triggered by EventBridge Scheduler.

Why Rewrite It?

I think a lot of us have been wondering when you'd pick durable functions over Step Functions. This project was small enough to rewrite quickly, but complex enough to exercise real patterns like API calls, branching logic, parallel execution, and multiple AWS services. Building the same thing both ways seemed like the best way to form my own opinion.

The durable function version is <a href="https://github.com/deeheber/durable-function-weather-site" target="_blank" rel="noopener noreferrer">here on GitHub</a>.

How the Two Approaches Compare

The workflow logic is identical in both versions. Same end result. But the developer experience is pretty different.

Step Functions

You define your workflow as a state machine, chaining together states like LambdaInvoke, Choice, Pass, and direct SDK integrations in CDK. The visual representation in the console is nice for understanding the flow at a glance, and it feels very "managed" in the best sense. The service handles state tracking, retries, and transitions for you.

The tradeoff is that CDK code for Step Functions can get verbose. Passing data between states and setting up error handling means understanding how Step Functions manages its JSON payload. You're giving up some control for the convenience of the service doing the heavy lifting.

Durable functions

With durable functions, you write your workflow as plain TypeScript. Wrap your logic in ctx.step() calls, use regular if statements for branching, ctx.parallel() for parallel execution. Need to call an AWS service? Just use the SDK like you normally would inside a step.

Here's the thing that surprised me: this version felt more natural to write. I wasn't thinking about "states" or "transitions." I was just writing code. Since it's TypeScript with a thin SDK wrapper, it also feels more portable. If another cloud provider or open source project adopted a similar checkpoint/replay pattern, the mental model (and probably a good chunk of the code) would transfer over.

Side by Side

Concept	Step Functions	Durable functions
Workflow definition	JSON/YAML state machine (ASL)	Plain TypeScript code
State management	Managed by the service	Automatic checkpointing via SDK
AWS service calls	`CallAwsService` task or direct integration	Regular AWS SDK calls inside `ctx.step()`
HTTP calls	`HttpInvoke` task + Connection resource	Standard `fetch()` inside `ctx.step()`
Conditional logic	`Choice` state	Plain `if` statement
Parallel execution	`Parallel` state	`ctx.parallel([...])`
Scheduling	EventBridge Scheduler -> Step Function	EventBridge Scheduler -> Lambda
Infrastructure	State Machine + Connection + Lambda(s)	Single Lambda function
Debugging	Step Function execution history (visual)	Durable execution history

The Gotcha That Got Me

Durable functions require a qualified ARN for invocation, meaning a published version or alias. When I first wired up EventBridge Scheduler, I was using the unqualified ARN and couldn't figure out why it wasn't working.

The issue? The CDK LambdaInvoke scheduler target uses functionArn under the hood, which for a plain Function is the unqualified ARN — no version or alias suffix. Durable functions reject unqualified ARNs entirely.

What ended up working was creating a Lambda alias in CDK and passing that to the scheduler target instead. Since an alias implements IFunction, the construct picks up its qualified ARN automatically.

If you're getting started with durable functions and things aren't behaving as expected, check your invocation ARN first.

So Which One Should You Use?

In my opinion, both approaches are solid for a workflow like this. It really comes down to developer experience.

If you prefer writing workflows as code, durable functions are going to feel great. The code reads like a normal application, the infrastructure footprint is smaller, and the patterns are more transferable if you ever need to move beyond AWS.

If you value visual debugging and a fully managed experience, Step Functions has the edge right now. The console gives you a step-by-step view of every execution with clear status indicators. Durable functions have a visual for execution history too, but I prefer how Step Functions handles it.

If you need Distributed Map for massive fan-out scenarios (up to 10,000 concurrent executions) or rely heavily on Step Functions' 220+ native service integrations, Step Functions is the better option.

AWS also has <a href="https://docs.aws.amazon.com/lambda/latest/dg/durable-step-functions.html" target="_blank" rel="noopener noreferrer">official guidance</a>. One thing I found interesting is they mention that hybrid architectures are totally valid, using durable functions for application-level logic while Step Functions handles higher-level cross-service coordination.

Wrapping Up

If you're curious about durable functions, I'd encourage trying it with a project you already have. You'll learn way more than a tutorial can teach you.

Both repos are open source:

Step Functions version: <a href="https://github.com/deeheber/weather-site" target="_blank" rel="noopener noreferrer">github.com/deeheber/weather-site</a>
Durable functions version: <a href="https://github.com/deeheber/durable-function-weather-site" target="_blank" rel="noopener noreferrer">github.com/deeheber/durable-function-weather-site</a>

I'd love to hear from anyone else who's been experimenting with durable functions. Let me know what your experience has been! 🚀

Let an AI Agent Do Your Job Searching

Danielle Heberling — Sun, 08 Feb 2026 09:12:03 GMT

Photo by <a href="https://unsplash.com/@markuswinkler?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Markus Winkler</a> on <a href="https://unsplash.com/photos/a-typewriter-with-a-job-application-printed-on-it-XKKuY4ottJ0?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

Quick disclaimer before I get into this: I'm no longer actively job hunting. I started building this project during my last job search and decided to finish what I started. Partly because I wanted to learn more about agents and partly because I think it might be useful for someone else out there. With that out of the way, here's the story.

The Problem

Ever since I started my last job search, one thing kept bugging me. I had a list of "dream companies" that I wanted to work at, and I'd manually go check their career pages every few days to see if anything new popped up. Sometimes daily. Sometimes I'd forget for a week and miss a posting entirely.

Here's the thing: job searching is already stressful enough without having to remember to check 10+ career pages on a regular cadence. I'd open a browser tab, search around, get distracted by Slack or email, and then forget where I left off. Multiply that by several companies and it becomes a real time sink.

I kept thinking there has to be a better way. What if something could just watch those companies for me in the background and send me an email when there's an opening that matches what I'm looking for?

That's when I decided to build it myself.

What I Built

The <a href="https://github.com/deeheber/job-search-agent" target="_blank" rel="noopener noreferrer">job-search-agent</a> is an AI agent built with <a href="https://strandsagents.com" target="_blank" rel="noopener noreferrer">Strands Agents</a> and deployed to <a href="https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/agents-tools-runtime.html" target="_blank" rel="noopener noreferrer">Amazon Bedrock AgentCore Runtime</a>. You give it a company name (and optionally a job title or location filter), and it searches the web for open positions at that company. Then it returns what it finds with links to the actual job postings.

You can also set it up to run on a schedule and optionally send you email alerts when a company is hiring. That's the part I was most excited about.

Design Decisions

Here's how the architecture works at a high level:

EventBridge Scheduler kicks things off on whatever cadence you configure. This triggers the agent running in AgentCore Runtime, which is where the Strands agent lives. The agent does its thing (searching for jobs at the companies you've specified), and if you've provided one or more email addresses in the environment variables and there is a job posting, it sends the results via SNS as an email notification.

The infrastructure is all defined in CDK (TypeScript), and the agent code is Python. I talked about why I use that particular combo in my <a href="https://danielleheberling.xyz/blog/strands-agent-template/" target="_blank" rel="noopener noreferrer">strands-agent-template post</a>, but the short version is that each framework is strongest in its native language right now.

Is this a bit over-engineered for what it does? Most likely. But I wanted to use the opportunity to build something I'd actually use while learning about agents and AgentCore. In my experience, the best way to learn new tech is to solve a real problem with it, even if the solution is fancier than it needs to be.

Challenges

When I first started building this, I took what felt like the simplest approach. I put instructions in the agent's system prompt to construct the career page URLs for each company. Something along the lines of "for company X, try going to company.com/careers and look for job postings."

This didn't work well at all.

The agent would guess at URLs that didn't exist, hallucinate job listings, and burn through a ton of tokens trying to figure out where the careers page actually was. Some companies have their postings on Greenhouse. Others use Lever, Ashby, or their own custom systems. There's no standard for this, and asking an LLM to guess the right URL is a recipe for frustration.

So I decided to build a tool for the agent to use instead. During that research, I came across <a href="https://www.tavily.com/" target="_blank" rel="noopener noreferrer">Tavily</a>, which is a search API built specifically for AI agents. Even better, Strands already had a pre-built Tavily tool available. I plugged it in, and the difference was night and day. Instead of guessing URLs, the agent now searches the web for actual career pages and job boards, then extracts the relevant information from those results.

This was a good reminder for me that agents are only as useful as the tools you give them. The LLM is great at reasoning and formatting responses, but it shouldn't be doing the heavy lifting of web searching through prompt instructions alone.

There was one more quirk worth mentioning. EventBridge Scheduler supports <a href="https://docs.aws.amazon.com/scheduler/latest/UserGuide/managing-targets-universal.html" target="_blank" rel="noopener noreferrer">universal targets</a>, which let you call pretty much any AWS API action directly without needing a Lambda function in between. You just give it the service ARN and the request payload, and it makes the API call for you. There's a <a href="https://docs.aws.amazon.com/scheduler/latest/UserGuide/managing-targets-universal.html#unsupported-api-actions" target="_blank" rel="noopener noreferrer">list of unsupported action prefixes</a> in the docs (things like get, describe, list). Interestingly, invokeModel is explicitly blocked, but invokeAgentRuntime is not.

So I set up EventBridge Scheduler to call the Bedrock AgentCore invokeAgentRuntime action on a schedule. And it works. The agent gets invoked, does its thing, and returns results. But for some reason, EventBridge Scheduler thinks the invocation failed. Every single time.

If you leave retries enabled, the scheduler will retry the invocation multiple times even though the first call succeeded. That means your agent runs (and costs you money) three or four times instead of once. Also you may get duplicate email notifications. Not great.

My workaround was to disable retries on the scheduler and set up a Dead Letter Queue (DLQ) to capture these "failed" invocations so I can monitor them. I'm honestly not sure if this is something I'm doing wrong or if it's a quirk on the AWS side that'll get fixed eventually. AgentCore is still pretty new, so it's possible this just hasn't been ironed out yet. If you've run into something similar, I'd love to hear about it.

The Outcome

After getting Tavily integrated, the agent started returning real job listings with real links. I set up EventBridge to run the searches on a schedule and configured SNS to email me the results.

The time savings were immediately obvious. Instead of spending 30+ minutes bouncing between career pages every few days, I'd get an email in my inbox with a summary of who's hiring and links directly to the postings. Even when nothing new showed up, it was nice to not wonder if I'd missed something.

A Note on Cost

If you decide to deploy this yourself, keep in mind that it will cost you money. Specifically:

Bedrock usage is the main cost driver here. Every time the agent runs, it's making calls to a foundation model, and those tokens add up. How much depends on how many companies you're monitoring and how often the scheduler runs.

Tavily has a free tier that's pretty generous for personal use, but if you're running searches frequently or monitoring a lot of companies, you could exceed it. Keep an eye on your usage.

I'd recommend starting with a low frequency schedule (maybe once a day or even once a week) and a small list of companies to get a feel for the costs before scaling up.

Everything else in the stack (AgentCore Runtime, EventBridge Scheduler, SNS) will likely fall within the AWS free tier for personal use. But "likely" isn't a guarantee, so check the <a href="https://aws.amazon.com/pricing/" target="_blank" rel="noopener noreferrer">pricing pages</a> against the usage you expect before deploying.

What I Learned

I think the biggest takeaway from this project is that building something you actually need is still the best way to learn new tech. I could have followed a tutorial to build a generic chatbot, but because this was solving a real problem for me, I was way more motivated to push through the frustrating parts.

If you're curious about agents but not sure where to start, I'd suggest finding a small, annoying, repetitive task in your life and trying to automate it. It doesn't have to be fancy. The learning happens in the process of figuring out what works and what doesn't.

Give It a Try

The full source code is on <a href="https://github.com/deeheber/job-search-agent" target="_blank" rel="noopener noreferrer">GitHub</a>. It includes instructions for running locally and deploying to AWS.

If you're currently job hunting and this looks useful, take it for a spin. If you find bugs or have ideas for improvements, open an issue or submit a PR. I'm curious to hear what other folks build with it.

Good luck out there 💪

A Strands Agent Template (For the Impatient)

Danielle Heberling — Fri, 23 Jan 2026 10:12:03 GMT

Photo by <a href="https://unsplash.com/@ttepavac?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Tanja Tepavac</a> on <a href="https://unsplash.com/photos/a-piece-of-a-puzzle-with-a-missing-piece-cWMhxNmQVq0?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

Ever since AWS announced <a href="https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/agents-tools-runtime.html" target="_blank" rel="noopener noreferrer">Bedrock AgentCore Runtime</a> back in July 2025, I've been experimenting with deploying agents to production. The hype around "agentic AI" is everywhere, but most examples I found were either too simple or way too complex for what I actually needed.

Here's what I wanted 👉 a way to go from zero to a deployed agent on AWS without spending three days on infrastructure setup. So I built a template that does exactly that.

What I Built

The <a href="https://github.com/deeheber/strands-agent-template" target="_blank" rel="noopener noreferrer">strands-agent-template</a> repo is a GitHub template that gives you everything you need to deploy a <a href="https://strandsagents.com/latest/" target="_blank" rel="noopener noreferrer">Strands agent</a> to Amazon Bedrock AgentCore Runtime. Click "Use this template" and you're most of the way there.

What's included:

✅ Production-ready CDK infrastructure with IAM, logging, and tracing
✅ Local development setup so you can test before deploying
✅ GitHub Actions for CI/CD
✅ OpenTelemetry observability built in
✅ Testing, linting, and formatting configured

Why Python + TypeScript?

I went with Python for the agent code and TypeScript for infrastructure. This isn't because I love context switching (I don't), but because each framework is strongest in its native language. Strands has great Python support and documentation, while CDK is cleanest in TypeScript.

As these frameworks mature, I might consolidate to one language. But for now, this combo gives you access to the best libraries and examples for both.

Quick Start

If you just want to see it work after setting up the AWS CLI and configuring credentials:

# Test locally
cd agent && source .venv/bin/activate && python src/agentcore_app.py
# Test in another terminal
curl -X POST http://localhost:8080/invocations -H "Content-Type: application/json" -d '{"prompt": "What is 42 * 137?"}'
# Deploy to AWS
cd cdk && npm install && npm run build && npm run cdk:deploy

The template comes with a basic example agent. Replace it with your own logic and you're done. Under 10 minutes from clone to deployed.

What You Actually Get

Here's what stood out to me after using this in a few projects.

Local testing that doesn't lie. You can run your agent locally before deploying anything to AWS. This matters more than you'd think. I've wasted too much time debugging issues that only showed up after deployment.
Observability without the extra work. CloudWatch logs and OpenTelemetry tracing are configured out of the box. When something breaks (and it will), you can actually figure out why.
CI/CD that works. GitHub Actions run your tests on every push. Both Python and TypeScript tooling is configured. It catches the obvious stuff before you deploy.

If You're Just Getting Started

If this is your first time deploying an agent to AWS, here's what I'd focus on:

Start by running the example locally. Don't deploy anything yet. Get familiar with how Strands agents work.
Then make a tiny change to the agent logic. See how it behaves. Break something on purpose and watch what happens.
Only then deploy to AWS. Use the CloudWatch dashboard to see what your agent is actually doing.

The template includes detailed deployment docs in the DEPLOYMENT.md file. I tried to write them for someone who hasn't used CDK or AgentCore before.

What's Next

I'm actively using this template for a few projects, so it'll keep evolving based on what I learn. If you find gaps or have suggestions, open an issue or submit a PR. The CONTRIBUTING.md file has guidelines.

This is meant to be a starting point, not a finished product. Take it, modify it, make it yours.

You can find the template on <a href="https://github.com/deeheber/strands-agent-template" target="_blank" rel="noopener noreferrer">GitHub</a>.

Let me know if you use it. I'm curious what works and what doesn't for other folks trying to deploy agents on AWS.

Use Social Media Mindfully

Danielle Heberling — Sat, 17 Jan 2026 10:12:03 GMT

I quit Facebook in 2020 when a former coworker was spreading misinformation about what was happening in Portland, OR. He'd never been there and had no plans to visit. I was literally living in Portland at the time, telling him what I was seeing firsthand, but that didn't matter to him. That was it for me. I miss it sometimes, but mostly I don't.

Here's what I've noticed since then: the heyday of social media feels like it's behind us. In my opinion, Facebook peaked in 2008. Back then, it was about connecting with friends, sharing actually interesting updates about our lives. Minimal ads. It felt genuine.

Now? Wannabe influencers everywhere. More ads and brand accounts in your timeline than content from people you actually know. Bots running campaigns to get engagement through false things or distortions of reality. It's exhausting.

But here's the thing: I'm not saying abandon social media entirely. I'm saying use it differently.

How I Think About Social Media Now

I'm not scrolling feeds endlessly anymore. No traps of getting lost in reels or stories. I use <a href="https://buffer.com/" target="_blank" rel="noopener noreferrer">Buffer</a> to schedule posts, which keeps me from even looking at a timeline. I check in with intention when I need to, then I'm out. This one's harder than it sounds, but it makes a real difference in how much time you lose to these platforms.

With that said, social media still works for connections. DMs are good. Having actual conversations in comments is good. Longer discussions where you're genuinely exchanging ideas? Even better. This is where I think the platforms still have value if you're intentional about it.

I try to share things that might help someone else. Good articles I've read. Things I'm learning. Mistakes I've made. If it could save one person some time or frustration, it's worth sharing. The stuff you've learned the hard way, the patterns you're seeing in your day job...not to build a personal brand or chase engagement metrics, but because someone else is probably dealing with the same problems.

If you're job hunting, LinkedIn especially can help you connect with the right people. It's not your whole career strategy, but it's a useful tool when you need it.

Here's where I think we've lost the plot though: we've forgotten that coffee with friends to catch up beats any social media interaction. Travel somewhere to see people you care about. Those face-to-face conversations are what actually matter.

What This Actually Looks Like

For me, this means I spend my time learning, reading, and building things. When I do post, it's usually because I want to hear what other people think about something I'm working through. Or I've hit a problem that took me way too long to solve and I figure sharing it might save someone else the trouble.

I'm not trying to go viral. I'm not optimizing for engagement. I'm definitely not checking how many likes something got.

The goal is simple: be helpful. Connect with people who are thinking about similar problems. Have real conversations, even if they happen in threads or comment sections. That's what matters and what I'm focused on in 2026.

Spend your time learning, reading, and building things. Use social media when it serves a purpose. Skip it when it doesn't.

What's your approach? I'd be curious to hear how other people are thinking about this stuff.

How You Do It Matters

Danielle Heberling — Wed, 19 Nov 2025 15:12:03 GMT

Photo by <a href="https://unsplash.com/@jack_anstey?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Jack Anstey</a> on <a href="https://unsplash.com/photos/aerial-photography-of-road-zS4lUqLEiNA?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

Looking back on my career, the connections and opportunities I value most weren't the result of optimizing for speed. They grew from taking the long view.

How you do it matters just as much as what you do.

Take professional networking, for example. I've found that showing up consistently to community events, contributing even when I don't need anything, has led to the most meaningful connections and opportunities. That might look like speaking at a meetup, volunteering to help organize one, reviewing someone's resume, or answering career questions in Slack. Similarly, I'm drawn to organizations where solving real user problems is at the heart of what we build.

I once worked at a startup where the CEO's departing words really stuck with me. This man was far from perfect, but I enjoyed working with him professionally and did it twice, mostly because of our alignment on what matters.

Here's what he said upon his departure.

As you continue to grow without me, I'd ask you to keep one thing in mind: the way we do it matters. We haven't gotten here because we're the cleverest copywriters, the fastest programmers, or the slickest sales people. We don't work sixteen hour days, and we're not ruthless or cutthroat. We're kind. We care about each other. We bring out the best of each other's strengths, and we bridge each other's weaknesses. We create opportunities to experiment and take risks. We're humble. And we're better for it.

Of course, pragmatic tradeoffs are part of the job. I can move fast when needed, whether that's shipping a quick fix to unblock progress or meeting a tight deadline. The key is being intentional: planning to address the root cause rather than letting temporary solutions pile up. It's about being honest with yourself and your team about what you're doing and why.

In an age of viral videos for quick wins, I know it can be challenging to play the long game...but I encourage you to consider it.

This approach has led me to opportunities and relationships I wouldn't trade for anything. If you're wondering whether it's worth it to take a more intentional approach - I'd say give it a shot and see where it takes you.

Learning Outside Your Specialty | Why I Got a Kubernetes Cert

Danielle Heberling — Sun, 12 Oct 2025 10:12:03 GMT

Earlier this month I took the exam for the <a href="https://training.linuxfoundation.org/certification/kubernetes-cloud-native-associate/" target="_blank">Kubernetes and Cloud Native Associate (KCNA) certification</a> and passed.

After years of working in AWS Serverless, this raised some eyebrows. People asked: why?

Here's the honest answer: I've passed on applying to jobs because I didn't know Kubernetes (K8s). That needed to change.

The Reality of Job Postings

Look at DevOps and platform engineering roles today. Many of them want AWS experience, serverless knowledge, AND K8s expertise. Not all, but enough that it's a pattern.

Specializing deeply in serverless has been valuable for my career. But at some point, gaps in your knowledge start closing doors. When I'm scanning job postings and thinking "I could do this role, except for that one requirement," it's time to consider filling the gap.

This isn't about chasing every new technology. It's about recognizing when a skill has become common enough in your field that not having it limits options.

What "Best Tool for the Job" Actually Means

I still believe in "serverless first." But here's what I've learned: the best tool depends on context.

At the end of the day, users care that what you build is reliable, works, and solves their problem. Unless they're an engineer or tech geek, they don't care if you used K8s or serverless. They care about the outcome.

Tech decisions are made by people. People make choices based on their experiences and what they know. A lot of teams use K8s. Whether they "need" it is a value judgment that depends entirely on your perspective and experience. That's not what this post is about.

What I've realized: if I'm working with a team that's chosen K8s, fighting against that to use serverless creates friction. Understanding their world helps me work with them effectively, even if I'd make different choices.

This isn't about one technology being better than the other. It's about understanding enough of both to collaborate with different teams and make informed decisions when I do have a choice.

K8s Is Complex, But Learnable

People aren't wrong when they say K8s is complex. It is. There are a lot of moving parts.

But here's what studying for the KCNA taught me: if you take the time to understand the components and how they fit together, it's not magic. It's learnable. Breaking it down into pieces makes it manageable.

The complexity isn't a reason to avoid it. It's just a reason to approach it systematically.

My Advice: Depth First, Then Breadth

If someone asked me "should I learn K8s?" a year ago, I would have said: get really good at one thing first.

Master the fundamentals of whatever stack you're working in. Build that depth. Then expand your toolbelt.

Trying to learn everything at once means you'll be mediocre at many things instead of excellent at one. Depth first, breadth later.

Disclaimer: this is what works for me. Your situation might be different, and that's okay.

That's what this K8s certification represents for me. Not abandoning serverless or becoming a "K8s person." Just adding another tool to solve different problems.

What's Next

The plan is to complete the <a href="https://cloudresumechallenge.dev/docs/extensions/kubernetes-challenge/" target="_blank">Kubernetes Resume Challenge</a> and build some personal projects. The goal is practical knowledge, not just cert credentials.

Honestly, it's been fun to be a beginner again.

I'd love to hear from you: Have you learned outside your specialty? What pushed you to do it, and what did you learn in the process?

Most Valuable When Least Visible | The Security Paradox

Danielle Heberling — Wed, 18 Jun 2025 10:12:03 GMT

Photo by <a href="https://unsplash.com/@flyd2069?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">FlyD</a> on <a href="https://unsplash.com/photos/pink-and-silver-padlock-on-black-computer-keyboard-F7aZ8G7gGBQ?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a>

The Problem

Tale as old as time: the friction between building out new features and the foundations necessary to deliver those features. By foundations, I'm referring to things such as but not limited to: security, testing, Continuous Integration/Continuous Delivery, observability, and Infrastructure as Code.

Recently I attended AWS re:Inforce, an AWS security focused conference. Speaking with fellow attendees it was very common for people ask some variant of "how can I get people in my organization to care about security?"

Disclaimer that I do not consider myself a security person™️, but I do champion security initiatives often. Security is everyone's responsibility. Sometimes in the real world people get excited about new features and forget about less visible things such as security.

Here are some of my thoughts in the form of unsolicited advice on how I attempt to navigate this.

1. Just Do It

If you work in an organization that gives you the freedom to allocate your time to projects beyond what is officially committed to in the sprint, just do it. Then once it's finished, you can share with your team what you did, how it works, and why it's important.

It is often faster to just do it rather than spend the same amount of time asking for permission. Especially at smaller early stage startups, delivering results speak louder than words.

2. View Yourself as an Enabler

Security is often (and mistakenly) seen as an obstacle to rapid feature development. I suggest reframing the conversation to highlight how security actually empowers the team to deliver features more quickly.

Many security measures may not seem urgent now, but by the time they are, it’s often too late. Focus on communicating the value of being proactive instead of reactive.

3. Learn Incentives

Incentives influence our perspectives and priorities. Invest time in understanding key stakeholders, especially decision-makers, to learn what drives them and what matters most to them.

This understanding will help you frame your arguments more effectively when proposing needs that don’t have the allure of a shiny new feature.

Bonus: Get Involved in the Community

As tech professionals, it’s easy to find ourselves working in silos within our organizations. But the more I connect with colleagues outside of my current company, the more I discover that the challenges I’m facing have often already been solved by someone else.

Why reinvent the wheel when we can learn from each other’s experiences - and share our own insights to help others grow? Engage in conversations, exchange ideas, and don’t hesitate to ask questions, even if they seem basic. Often, those “simple” questions lead to the most valuable discussions and solutions.

In person conferences aren't your thing? Shameless plug to check out the Believe in Serverless Community on Discord!

Closing

Ultimately, trust is the foundation of successful software. When it’s present, it’s invisible; when it’s missing, it becomes impossible to ignore. Once broken, trust is incredibly difficult to rebuild—especially for early-stage startups still working to establish their reputation. If users can’t trust your product, they won’t use it.

I’m not claiming to have all the answers when it comes to security, but I hope sharing these thoughts sparks reflection or helps someone else feel less alone in facing these challenges. Your efforts to prioritize security truly matter, even when they go unseen. Keep pushing forward and championing the importance of strong foundations.

I’d love to hear from you: What strategies or insights have helped you balance the tension between shipping new features and building secure, reliable products? <br /> <br />

AWS Fargate on a Budget

Danielle Heberling — Tue, 18 Feb 2025 11:12:03 GMT

Photo by <a href="https://unsplash.com/@sharonmccutcheon?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Alexander Grey</a> on <a href="https://unsplash.com/photos/focus-photography-of-person-counting-dollar-banknotes--8a5eJ1-mmQ?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a>

The Business Problem

At the day job, there's a new application that needs to be deployed in a manner that meets these requirements:

the code is server side and it needs to run longer than 15 minutes
this is an internal use app, it can tolerate occasional disruptions
non-engineering stakeholders should be able to access this application
the application will be used by fewer then 10 people and there will be periods of time where no one will be using it
it is faster to get approval if we use Amazon Web Services, because we currently use AWS heavily

A Solution

For non-disclosure agreement and security reasons this is a variation of the solution, not the exact solution.

It includes the following:

One Fargate Spot instance behind an Application Load Balancer
Two EventBridge Scheduler schedules that turn the Fargate Spot instances on at a specified time and off at a specified time

Disclaimer: this is not the only way to accomplish this.

Architecture Diagram and Example Code

Here's the high level architecture diagram of this solution.

Example code with instructions on how to deploy this into your AWS account, can be found in this GitHub repo.

Disclaimer: in this example, we are using a demo Docker image. If you are setting up an application that contains information that shouldn't be open to the public internet, ensure that authentication is in place. Authentication has been intentionally omitted, because it is not the focus of this article.

Let's zoom in a bit closer on the two major components of this solution:

Fargate Spot
EventBridge Scheduler

Fargate Spot

With Fargate there are two capacity provider types: Fargate and Fargate Spot.

Fargate is the standard which gives you on-demand access to containerized compute.

Fargate Spot is similar to Fargate, but cheaper (advertised as up to 70% discounted) and can be interrupted by AWS. The reason for this is because AWS operates at a massive scale and lots of times there are instances available that will run and cost AWS money regardless. In order to make money off of this extra capacity, AWS offers this extra capacity as Spot instances at a discounted rate. Because AWS might need this capacity back as demand rises, they reserve the right to give you a two minute warning before shutting down your instance to put it back into the regular Fargate on-demand pool.

The launch blog post has an excellent overview for an AWS official description.

Launching a Fargate Spot instance involves 1. launching it and 2. setting up the container to handle a graceful shutdown given a two minute warning from AWS.

Launch the Instance

The example code uses the ApplicationLoadBalancedFargateService CDK construct.

You can specify one or multiple capacity providers with the capacityProviderStrategies property with weights. The higher the weight, the more often that launch type will be utilized.

The example code sets things up to prefer Spot, but to use a regular Fargate launch type in case there is no Spot availability.

capacityProviderStrategies: [
          {
            capacityProvider: 'FARGATE_SPOT',
            weight: 50,
          },
          // Backup in case FARGATE_SPOT is not available
          {
            capacityProvider: 'FARGATE',
            weight: 1,
          },
        ],

Handle Graceful Shutdowns

In the example code, this is done in two places:

Adds a stopTimeout to the container on the Fargate Task for 120 seconds (needs to be <= 2 minutes)
Sets the deregistration delay on the Application Load Balancer at 30 seconds to give the instance time to separate from the Load Balancer before terminating (also needs to be <= 2 minutes)

EventBridge Scheduler

In the example, there are two EventBridge Scheduler schedules - "up" and "down".

This is accomplished by using an AWS CLI command to set the desiredCount on the ECS Service to either 1 (on/up) or 0 (off/down). Here's the official documentation for the command.

If this command were run on the command line via the AWS CLI it would look something like:

aws ecs update-service --cluster <cluster-name> --service <service-name> --desired-count <desired-count-int>

For this app, we're using EventBridge's Universal Targets feature. Universal Targets allow you to run most (not all - see the link to view unsupported commands) AWS CLI commands directly in an EventBridge Schedule without the need to add (and pay for) compute (such as Lambda).

In our example, we have one Schedule that sets the desiredCount to 1 (on/up) at 9am PT Mon-Fri and another that sets the desiredCount to 0 (off/down) at 5pm PT Mon-Fri.

These cron expressions can be adjusted...be sure to ask your stakeholder(s) when they plan to use the app to ensure it is available.

Summary

This was a walkthrough of a solution that allows us to run a Fargate instance on a budget. The solution utilizes Fargate Spot and EventBridge Scheduler to periodically shut down the Fargate Tasks during periods of no usage.

The example code repository can be found on GitHub.

What tips and tricks do you have for saving money with Fargate?

Ephemeral Jobs Longer than the Lambda Timeout

Danielle Heberling — Thu, 07 Nov 2024 15:12:03 GMT

Photo by <a href="https://unsplash.com/@cadop?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Mathew Schwartz</a> on <a href="https://unsplash.com/photos/equestrian-riding-horse-at-daytime-5qRWQEdK7Sg?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a>

The Problem

There's an ad-hoc job that runs in the background that doesn't have a client waiting for a synchronous response.

You could run this in AWS Lambda to save money. A benefit with an ephemeral job run is that AWS only charges for when the Lambda is running.

Problem is that this specific job runs longer than the Lambda timeout (at the time of writing this it is 15 minutes). We could write extra logic to make it work, but that adds unnecessary complexity.

You still want to run this ephemerally to save money and resources...so what can you do?

A Solution

One option is to use ECS run-task with a Fargate launch type.

AWS Resources Needed:

An ECS Cluster - a logical grouping of tasks or services (in my example we're using the default cluster that comes with new AWS accounts)
A Task Definition - a blueprint for your application which contains one or more containers and parameters to run
A Container - a lightweight, standalone, executable package of software that includes everything needed to run an application. This is commonly used with Docker.

Once those resources are deployed, the container can be triggered to run on demand with ECS run-task. Once the code is done running and the container exits, then the running ECS Task will disappear. The benefit is that you'll no longer be charged money for a running Fargate task.

There's a few ways to run an ECS task on demand:

AWS CLI
AWS SDK (could invoke from within Lambda etc)
In a Step Function state
EventBridge Scheduler
AWS Console

Here's an example of what this command looks like via the AWS CLI:

aws ecs run-task \
  --cluster default \
  --task-definition my-task \
  --launch-type FARGATE \
  --network-configuration "awsvpcConfiguration={subnets=[subnet1,subnet2],securityGroups=[securityGroup1],assignPublicIp=ENABLED}"

Show Me the Code

There's a full code example of how to deploy these cloud resources and invoke the deployed task via ecs run-task with the AWS CLI.

Take a look at the README file for directions.

Bonus

There are a few more benefits to ECS run-task that I won't mention, but one notable one is the ability to override specific things in the task definition.

For example, if you want to override the CMD defined in the Dockerfile used to build the Container Image for the task, you could do this by adding the --overrides flag.

aws ecs run-task \
  --cluster default \
  --task-definition my-task \
  --launch-type FARGATE \
  --network-configuration "awsvpcConfiguration={subnets=[subnet1,subnet2],securityGroups=[securityGroup1],assignPublicIp=ENABLED}" \
  --overrides '{"containerOverrides":[{"name":"my-container","command":["npm", "run", "migrate"]}]}'

I've seen this used for one off commands that need to run every now and then such as database migrations, though I'm certain there are other good use cases. Here's the full list of what can be overridden.

Special thanks to Chase Douglas for giving me this idea. 🙌🏻

Playdate Review

Danielle Heberling — Sat, 28 Sep 2024 05:12:03 GMT

What is a Playdate?

The Playdate is a small hand held game system with a crank, buttons (D-pad, a, b), and a small black and white screen (similar to e-paper).

With dimensions of 76 × 74 × 9 mm, the console can easily fit into most pockets for on the go game playing.

Panic is the company that created the Playdate. You might have heard of Panic from playing Untitled Goose Game or Firewatch. Or you might have used Mac apps such as Transmit or Coda (renamed to Nova).

The CEO and co-founder of Panic gave an in depth talk about the story behind the creation of the Playdate which is entertaining and informative if you would like to go deeper.

Nice!

These are some things that I enjoyed about the Playdate:

You get 24 free games (also called "season one") 2 games delievered each week until you have them all. It's delightful to see that notification light that there's new games to play at the beginning of owning this game system.
Games aren't limited to their catalog, you can also build your own games using their SDK or side load games from places such as itch.io.
The crank as an input device is surpisingly delightful and unique.
Most games have enhanced details. We're talking beautiful storytelling, visuals, and music.
This game system is minimally connected to the internet to allow downloading of games, but otherwise there aren't distractions that would exist on a smart phone.
The battery life has been great so far. Marketing copy for this product boasts 8 hours when playing games and 14 days while in standby clock mode. This has matched my experience. They also use a generic usb-c charger as well, so it's likely you already have the plug/cord for it (though it comes with one).

Oh.

These are some things that I think could be improved with the Playdate:

This game system does not have a backlight, so if you don't have good lighting it will be hard to see the screen.
I'm left handed and would love to have a "lefty" version with the crank on the left side of the game system.
Right now the only body color is yellow which is great, but I would like more color options especially when having multiple Playdates in my house.
There is not a way to shut it off. It will go in standby clock mode which is fine, but I would like the ability to fully shut the game system off without needing to let the battery die.
Right now for sound, it is wired headphones only. Looking through their website it appears that bluetooth is likely coming soon with a software update. To my knowledge the wifi chip they have in the game system should also double as bluetooth, so I don't think a hardware update is needed for this.

Where and how to buy

I'm seeing a few resellers, but the official place to buy and where I bought one was from the official playdate shop.

My opinion

I would highly recommend buying this game system. The Playdate is a lovely distraction from a world full of advertisements, pop ups, and notifications.

As a kid, I could sit for hours and play video games, but as an adult I find that I do not have much time to devote to playing video games. The games on this system are perfect for that. You pick up the game system, play for 15-30 min here and there plus the controls aren't complicated...so you're not spending time trying to remember how to play the game.

Also anything Panic creates, I know will be high quality and well thought out.

Side story

Panic holds a special place in my heart. Back when I recently relocated to Portland, OR I was looking to get out of my current company and saw that they had an open customer support role. I was currently working in customer support, but moonlighting by taking night/weekend classes at a local code school to switch into being a software engineer. I went into their office and did the full interview round (including an interview with their CEO, Cabel Sasser). The next day I received the nicest rejection email from Cabel stating that they really liked me, but think that I would be better suited to becoming a software engineer and recommended that I continue down that path.

Anyway I highly recommend the Playdate and anything that Panic creates, because I know it will be high quality in terms of the aesthetics and story telling. <br /> <br />

Devs are Not the Target Market for AI

Danielle Heberling — Fri, 30 Aug 2024 12:12:03 GMT

Background

The year is 2024 in the tech industry. Layoffs and downsizing are everywhere. It is a very challenging job seeker market. There are lots of recent code school and computer science degree graduates seeking their first software engineer job. Amidst all of this noise, there is massive hype about "generative AI."

More on Artificial Intelligence (AI)

In my part of the world, software engineers are tired of hearing about AI and feel like it is mostly a solution looking for a problem. Everyday users I interact with also echo these sentiments.

To contast this, I'm seeing two groups of people that are very excited about AI 👉🏻 people selling things and business executives/investors at various companies.

The motivations of people trying to sell you something is pretty straighforward. They want to sell things to make money for their business.

The executives/investor's motivations are less clear. Here's my theory, many of them view employing humans as a liability both because dealing with humans can be messy but also because paying employees a salary is expensive and they think that AI can be a substitute.

A harsh truth is that many businesses exist to make money and to be profitable. If they no longer need to have the expense of paying employee salaries, this can help them move closer to that goal. Most companies do not exist for the purpose of providing jobs to people.<sup>1</sup>

My Opinion on AI

I have concerns over AI ethics. Bias in the models, creator's work getting stolen for use in AI models without compensation, and environmental concerns of needing to run many servers to train these huge models. Many members of the United States government do not understand how technology works, so I don't trust that they can effectively regulate.

I think that these tools are quite good, but they're not sophisticated enough yet to fully replace human software engineering labor on a wider scale.

I have concerns over the ability for human or AI software engineers to be able to effectively troubleshoot and maintain unvetted AI generated code that gets pushed to production.

I am excited about the potential for AI to do boring and mundane tasks for us humans, so we can focus on imaginative creativity. An example: I'd like AI to do the dishes for me, so I can write my next novel, paint my next portrait, or compose my next song. I haven't seen many examples of this yet.

My Opinion Ultimately Doesn't Matter

What does matter if you want to be employed as a software engineer in this industry is the opinion of those who are in positions of power that make the budget and hiring decisions.

Unsolicited Advice

If you want to be hired and maintain a job in this industry as a software engineer, you need to figure out the motivations of people in positions of power and do the work to be able to effectively and genuinely sell yourself to them.

Some Ideas

Here's some themes I've seen lately with people in positions of power at various companies. Note that my bias leans toward small/medium size businesses since that is where the majority of my social network is employed. Your experience may be different.

They want to do more with less

I would advise being curious. When something isn't working as expected, be willing to adventure outside of your comfort zone to figure why, fix it, and learn new things.

On the web development front, more places desire to hire "fullstack" engineers. Just doing frontend is not enough for many companies.

They want to deliver solutions

Focus on shipping. It's not enough to only write code, you need to know how to get a full working application into production that satisfies all product requirements. You also need to know how to troubleshoot and diagnose bugs that happen to your users in production.

It's also important to be someone who can be trusted to follow through with what you say you will deliver. If you get stuck on something, it's your responsiblity to ask for help to get the assistance you need to follow through all the way.

They're excited about AI and some want to replace us devs with AI because it is cheaper than paying a salary

I would advise investing time in learning and using these new AI tools. Learn what they're good at and use them to accelerate your output. Learn what they're bad at and invest the time in learning how to do these things well by yourself. Learning those skills is a good way to emphasize your value to executives as to why they should hire you and pay you a salary instead of using AI.

Closing

Anyway, these are some thoughts I've had over the past few months. I'm just another person on the internet with opinions, giving unsolicited advice. What you do or do not do with it is up to you. I welcome polite engaged feedback and conversation on this topic.

<br /> <sup>1</sup> Note that this is not commentary from me saying that I like the status quo. This is a comment plainly stating the status quo.

I'm Worried About Generative AI

Danielle Heberling — Sat, 04 May 2024 22:12:03 GMT

Photo by <a href="https://unsplash.com/@dearseymour?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Ksenia Makagonova</a> on <a href="https://unsplash.com/photos/opened-white-wooden-drawer-bngKirnA1EE?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a>

I'm worried about generative AI. I say this as someone who uses AI to help with work tasks daily. I do believe there are some neat things that AI can do and there is high potential for it in the future as it improves, but I have concerns.

No this post isn't about the common objections I've been seeing in the discourse. Things like bias in the models, hallucinations, and people's jobs being replaced. Although they are very important things to monitor.

I'm concerned with AI monopolizing the focus of companies.

Take the big tech companies for example. There are reports everywhere about multiple layoff rounds but also news of higher rates of investment (both money and people resources) in AI than ever before. These large companies have huge ecosystems that many folks depend on. Does this mean since the main focus is AI that there will be less focus on their non-AI products and services? Does this mean that service reliability will degrade as time goes on because engineers who previously responded to outages were moved to teams working on AI or were laid off?

On the other end of the spectrum we have tech startups. I'm seeing many chatbots being built and their marketing tells me this is the solution to my problems (that I didn't know I had). I'm excited that this opens up opportunity for new ideas and businesses to take shape, but when the core of a product is centered on what is essentially a black box that might be very wrong sometimes...this doesn't feel like a good business model to me until the tech matures. My overall impression is that AI is a magic word that these companies can use to attract more customers who like shiny new things and also to attract venture capital funding.

I've also seen lots of companies throwing AI at things when it wasn't needed. An example: a company who will remain nameless was demoing a new tool that uses AI to help users troubleshoot error messages that surface in their app. A developer watching the demo yelled out "or you could consider writing better error messages that users will understand instead of bringing AI into this."

These are my opinions based off of observation of the world around me. I do not have data to present as evidence. This is all based off of vibes. I sure hope my vibes are wrong, that I'm overreacting to all of this, and that I should take off my tin foil hat.

Time will tell.

AI has promise and it does have the potential to be helpful for specific types of tasks, but let's not forget that we should attempt to solve problems that already exist before bringing in tools that have the potential to cause even more problems.

What are your thoughts?

Tips for Getting Started in Cloud

Danielle Heberling — Fri, 12 Apr 2024 09:12:03 GMT

Photo by <a href="https://unsplash.com/@billy_huy?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Billy Huynh</a> on <a href="https://unsplash.com/photos/cloudy-sky-at-daytime-v9bnfMCyKbg?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a>

Who am I?

I graduated from a code school in 2016 in the United States and have been employed as a software engineer ever since. Because I like to give back to the communities that helped me get started, I've done paid contracts and volunteered in positions where I mentor folks who would like to take a similar path.

After years of sharing my knowledge in the wider AWS community, I was recognized as an AWS Serverless Hero in 2023.

My goal in writing this

I often surround myself with people in the code school community who have expertise in "Full Stack JavaScript" (it's mostly Frontend focused from my observation). Because of this, I am often asked the questions like "how can I get started in Cloud" or "how can I learn AWS." It is my hope that this post can serve as a reference for folks who ask me those questions.

My biases

We all have biases, myself included. I encourage you to do what you think is best for you. Do not take my advice as an absolute, "I must do everything on this list."

Cloud-wise I have mostly worked in AWS, so my opinions and advice will likely be AWS centric. It is my personal opinion that I wish that there was more Cloud competition (specifically one that offers the same uptime/choices of Serverless services AWS offers but with a better developer experience), but in my part of the world most of the jobs available are fully AWS or AWS heavy in terms of Cloud usage. This is why I've chosen to focus on this Cloud and not the others.

I also started my career as a "Full Stack JavaScript" developer and thus was originally very Frontend focused. At the time, I mostly cared about what the customers experience and the implementation details (while important) were less interesting to me. To that end, I was drawn to using managed services that AWS offers (sometimes called "Serverless") rather than rolling my own by using solutions such as Kubernetes or writing lots of Dockerfiles (sometimes referred to as "Cloud Native").

Should I get cerfitications or do hands-on projects?

This is often the first question I am asked and many folks (some with monetary incentives) have shared their advice on the internet. Here is my biased opinion.

As a Frontend focused engineer, I had no idea what any of the AWS alphabet soup of managed services did or their names. I started my journey by obtaining the AWS Certified Solutions Architect - Associate certification.

I used my knowledge gained from studying for that exam as a jumping off point to start building my own projects. Lately, I've mostly been learning new things through hands on projects.

I view certifications as the "theory" and hands on as the "practice." Getting certifications without the hands on practice will not get you the job. From my experience most employers care more about what you have done and can do hands on inside of AWS.

For me a 30% focus on certifications (because my employer and various AWS community programs will pay for the exam) and 70% building side projects for fun is what has worked the best for my learning.

If I'm curious about an AWS service that I've never used, I will build a side project focused on it to learn. These side projects do not need to be public or used as a portfolio piece for job interviews. With these side projects, I like to take advantage of having a smaller codebase and a controlled environment (my own AWS account and not a production app for work) to experiment and learn.

Infrastructure as code

Lots of certification prep material and getting started content will encourage you to click through the AWS console to create your Cloud resources. This is informally known as "click ops."

I would highly recommend you don't do this and learn to use an infrastructure as code (IaC) tool while building side projects for learning. This is much closer to how things will look once you have a job and are working on a team.

There's lots of arguments across the internet about which one is the best. Some folks weirdly get cult-ish about it. My opinion is to pick one and use it! The skills are transferrable across the different IaC tools from my experience.

Some commonly used ones with AWS are:

AWS Application Composer might also be a good starting point because it is a drag and drop interface for your cloud resources, but you also have full access to the IaC that it generates.

Along the lines of IaC, you should also look into CI/CD. There are lots of tools to accomplish this, but one that I've been using lately on the job has been GitHub Actions. Again, I recommend you find one and use it. The principles and higher level ideas are transferrable across different CI/CD platforms.

Recommended resources

Here's a list of resources that I recommend. As new material comes out, I will do my best to keep this updated. Disclaimer that these are all things that I've used or are created by folks whom I respect. No one paid, sponsored, or asked me to add their content to this list. Also note that some of these resources have Discord communities where you can interact with fellow learners and ask for help.

Free resources

Mix of free and paid resources

Paid resources

Closing

I hope you found my biased opinion on this topic helpful! Cloud and AWS are big things to learn, so be patient with yourself and join some communities where you can ask for help. I has taken me years to learn what I know and I still don't know everything.

Do you have any getting started in the Cloud resources you'd like to share? Please do!

Good luck out there. I can't wait to see what you build!

The Collab Lab Team 66 Recap

Danielle Heberling — Sun, 07 Apr 2024 22:12:03 GMT

Overview

The Collab Lab is a non-profit organization that runs a program where early career developers can apply to work together on a team to complete a project led by mentors who work in the field.

The Team

Participants

Mentors

The Project

We were tasked to build a "smart" shopping list over the course of 10 weeks. During this time there was lots of pair programming, async communication over slack, code review, demos, and retrospectives.

Features

Login and customized views based off of whom logged in
Create a new list
Add an item to the list and set the urgency level (soon, kind of soon, not so soon)
Delete an item from the list
Mark an item as purchased
Search a list for an item
Share a list with another user

Tech Stack

JavaScript React Flowbite Tailwind Firebase - Auth, Database, and Hosting Vite

The Finished Project

Login Page

Landing Page - select a list to view/edit

A Specific List Page

The source code for the app can be found on our GitHub repo.

You can view this app live at https://tcl-66-smart-shopping-list.web.app/.

Closing

We had a great 10 weeks learning from each other and are looking forward to continuing to build fun projects.

If you're looking to hire early career developers, I would highly encourage you to take a look at these folks' LinkedIn profiles.

How To Become An AWS Hero

Danielle Heberling — Wed, 03 Apr 2024 05:12:03 GMT

Photo by <a href="https://unsplash.com/@gabrielbassino?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Gabriel Bassino</a> on <a href="https://unsplash.com/photos/yellow-neon-light-signage-zEawlLdVloo?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a>

Ever since I was named an AWS Hero, lots of people have been asking me how they can become an AWS Hero.

In my opinion, this is the wrong question to be asking.

I never had the goal to become an AWS Hero. I did and will continue to do what I'm doing now regardless of the AWS Hero title. Don't get me wrong, I am very humbled and honored to have been recognized as one by AWS, but this is not why I do what I do.

In my opinion, the correct question to be asking is what are the rough edges and challenges in the current ecosystem and how can I share my knowledge and experiences to make it easier for others?

For ideas and inspiration, I recommend looking at what AWS Heroes and AWS Community Builders are doing. Also, remember that you can do something completely new too!

You don’t need permission or a fancy title to help others and to make a difference. If you aren't sharing your knowledge and experiences in public yet and would like to, please join us!

I'm looking forward to seeing what you share.

Disclaimer: This post reflects my personal opinions. It may or may not be representative of the opinions of other AWS Heroes or AWS. You will need to ask them for theirs. 😀

Dear New Code School Grad

Danielle Heberling — Thu, 22 Feb 2024 22:12:03 GMT

Photo by <a href="https://unsplash.com/@rutmiit?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">RUT MIIT</a> on <a href="https://unsplash.com/photos/people-in-black-academic-dress-standing-on-green-grass-field-during-daytime-hpRGrfOIybc?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a>

Over the years, many people have come to me for advice on doing a career change into software engineering. Here are my thoughts written down relevant to doing this in the year 2024 in the United States.

Who am I?

I graduated from a code school in 2016 in the United States and have been employed as a software engineer ever since. I've held both formal and informal mentor titles. The formal ones were through side jobs working for code schools or being a mentor for nonprofits that help folks break into tech. The informal ones are via warm intros from network connections with recent code school grads seeking help.

My goal in writing this

I want you to succeed. We need more people in this industry that bring diverse life experiences to make our products truly reflect the public that we serve.

There are a lot of folks on the internet and in real life that want to sell you the idea that a career switch and learning how to code is easy. I'm here to tell you it is not.

Trends in 2024

The hiring market has evolved since I was looking for my first software job in 2016. It is much more challenging now. Here's a list of trends that I'm seeing in my part of the industry:

More code schools exist than did in 2016
Most of these code schools teach full stack JavaScript with an emphasis on the frontend (react in particular)
Due to higher enrollment in these programs 👉🏻 more people are looking for their first job as a full stack JS or frontend developer
Many hiring managers are only interested in hiring people who have experience. It doesn't seem good enough to be "senior" anymore, lots are looking for "staff" or higher levels (what these levels actually mean can vary from company to company)
Due to "macroeconomic conditions" many companies are conducting layoffs including software engineering staff

Unsolicited advice

It is harder than ever to break into tech. Know that a large part of finding a job in any market is about being in the right place at the right time. Part of it is putting in the work to land that position...but if the job doesn't exist then you don't have a chance to land it. Be kind to yourself and know some of this is outside of your control.

In my opinion, the most important thing to do is find ways to differentiate yourself. How are you different from the thousands of applications to this one open position and why should they hire you over the other applicants? Due to the high number of career changers seeking jobs, the story of a career change is no longer enough to make yourself stand out.

A good way to start is by looking at job postings and taking note of specific technologies and positions that are in high demand. If you don't know those things 👉🏻 put in the work to learn it. We live in a golden age where there is tons of content (paid and unpaid) online.

One caveat: there are lots of "influencers" trying to sell you something. Do your homework to investigate that individual or organization's background before purchasing anything. Try to find some LinkedIn profiles or ask folks in your network for a second opinion. It is possible they lied on their LinkedIn, but if they are qualified there should be artifacts somewhere in public spaces. Just because someone is loud and all over social media, it doesn't necessarily mean that they are an expert.

Back to in demand skills. This is my highly biased opinion, but in my part of the industry there is high demand for people who work with the cloud. In all of my recent jobs, there were lots of frontend engineers and not enough people to do things on AWS side of our stack. This might look like a software engineer that builds apps that integrate with cloud managed services or it could be more of a "DevOps" type of person. (Side rant: DevOps should never be a job title in my opinion as it is a philosophy that everyone should follow despite their official title, but that could be another post.)

A mistake that I see new graduates do is focus too much on making their LinkedIn profile and resume perfect. It is important to have a good online presence and resume, I'm not debating that. But equally important is finding people (specifically people who are hiring/have influence in hiring decisions) to read your profiles/website/resume. From my experience, the best way to do this is by networking. I know most career advice says this, but it is true. All of my software engineering jobs have come from some form of networking and I'm an introvert. It will be very uncomfortable, but if you want this badly enough you will need to acknowledge that unfortunately we live in a world where this is what you have to do to get people with influence to pay attention to your candidacy.

Ideally this would be done via a warm introduction from a common connection, but don't worry about sending messages to people you don't know. As long as you're directly asking for what you want and respecting people's time, they will most likely reply and want to help you. Worst case is that they don't reply. View relationship building as a long term strategy and not as a one time transaction. You never know if you might one day be able to help this individual in return. Anecdotally, there have been many times when people whom I've formerly mentored were able to help me out too later on. We're all in this together as a community and remember to give as much as you take in the long run.

Another mistake that I see new graduates do often is they start a conversation with a hiring manager by stating they're a beginner or looking for a junior role. In this hiring climate, many hiring managers are seeking people with prior experience (see bullet point above), so starting out the conversation this way will most likely result in them losing interest in your candidacy. I'm not suggesting you lie or misrepresent your experience at all. I'd recommend talking about some things that you've built, the the technologies you've worked with, what parts of tech you're excited about, and what type of role you're looking for. If the question of previous professional experience comes up in conversation then tell the truth. "Junior" and "entry level" can mean different things to different hiring managers. Allow them to make that judgement after getting to know you a bit and hearing about what you've been working on.

Closing

Keep in mind that some of these thoughts are very specific what's going on in the year 2024 in the United States, so if you're reading this in the future or from a different location some of it might not be relevant.

I am not trying to sell you anything and really want you to succeed. This post is a compilation of my thoughts that many in my network have asked for, so I hope you find me publicly sharing this helpful.

It is totally up to you if you want to follow this advice, because at the end of the day I'm just another random person on the internet with opinions.

The Serverless Mindset and Infrastructure as Code

Danielle Heberling — Tue, 05 Dec 2023 12:12:03 GMT

Originally, I wanted to blog about my favorite re:Invent announcements. But as I started, I saw a higher level theme and decided to write on that instead.

Disclaimer that this post contains my individual opinions and yours might differ. I welcome your polite/engaged feedback and encourage you to continue this conversation.

Relevant Announcements

A Lightbulb Moment

My time spent as a software engineer at $day_job is mostly writing Infrastructure as Code (IaC). When reflecting on the Serverless mindset, I believe that some aspects of this mindset conflict with how I spend most of my time at $day_job.

✅ Pros of the status quo: writing IaC is repeatable, scalable, fun, and I'm good at it! 🙂

🚫 Con of the status quo: I find myself wanting to spend more time focusing on providing business value and less time thinking about tooling or specific settings for my cloud resources.

Then a thought occurred to me: What if we didn't have to write IaC? Why not have a deployable diagram of the application instead?

This is where I think tools such as AWS Application Composer and AWS Step Functions Workflow Studio are headed in a positive direction via their drag and drop visual interfaces.

To a Brighter Future

Werner Vogels said in his keynote that "the most dangerous phrase in the English language is: 'we've always done it this way'." I think this applies to how we've historically approached IaC with Serverless development.

Anecdotally, I've noticed that some experienced builders are ignoring updates to tools like this because they already know how to write IaC and think they don't need a visual interface. I politely disagree.

To me, being able to more quickly collaborate, share, and deploy cloud resources is a huge win for everyone, regardless of experience level or job title. The primary focus can then be providing business value rather than figuring out how to write IaC for a specific cloud resource.

We have so much more to offer than expertise in the tools that we use. Embrace the Serverless mindset and demand a brigher future. 😎

EventBridge Emoji Event Patterns

Danielle Heberling — Sun, 24 Sep 2023 07:12:03 GMT

Photo by <a href="https://unsplash.com/@denic?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Denis Cherkashin</a> on <a href="https://unsplash.com/photos/qIKSsOMIhpM?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

Background

The other day, I was answering some questions a coworker had about EventBridge. He then jokingly said "I wonder if I can use emojis for an event pattern because they are UTF-8."

After that conversation, I couldn't stop thinking about it. So I threw together a quick CDK stack and it works!

Try it yourself

I've made my quick CDK stack public on Github. You're welcome to test with mine.

This stack contains

A rule with emojis only in the Event Pattern on the default event bus
A Lambda as the target of the rule that will be invoked and logs the event

Steps

Clone the repo
Configure your AWS credentials locally if you have not yet
npm i && npm run cdk deploy to deploy the stack
Put events on the default event bus with either the CLI, and AWS SDK, or in the AWS console. The event pattern is detailType: ['💩'] and source: ['🐶'].
Check the Lambda's CloudWatch logs to see that it has been invoked

Closing

You can also update the code to contain other emojis as well. From my testing, I found having multiple emojis in a single event pattern works as well.

Not 100% practical in all cases, but thought I would share these findings publicly. Now go forth and update your EventBridge event patterns. 🎉

My Personal Blog Site's CI/CD

Danielle Heberling — Mon, 17 Jul 2023 22:10:03 GMT

Photo by <a href="https://unsplash.com/@dancristianpaduret?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Dan Cristian Pădureț</a> on <a href="https://unsplash.com/photos/XC7lc8biINg?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

Goals

In 2018, I set out to create this blog. Medium and other blogging platforms existed; however, I wanted to have more control and ownership of my content. To me, it's fine to syndicate blog blog to other places for more reach, but I wanted my own space on the internet.

I also had the other following goals in mind:

Low maintence
Low cost (or free)
Use technology that I wanted to learn
Write blog blog in a transferrable format, such as markdown

Tech Choices

In 2018, the hype of JAMstack and Gatsby was in the air. Upon taking a closer look at Gatsby, I learned that it utilized GraphQL (tech I didn't know at the time and wanted to learn), React, and blog blog were written in markdown. After choosing Gatsby, I did some searching for the easiest way to deploy it and landed on Netlify.

To this day, my blog is still Gatsby hosted on Netlify and you can view the code source on GitHub.

Implementation

CI/CD

When a new site is created on Netlify, you can link it to a GitHub repository. You then have the options to set up CI/CD to perform build, deploys, or previews whenever you push or merge to a specified branch.

For my site, I decided to keep it simple and stuck with the defaults. Whenever I put up a Pull Request against the main branch, Netlify runs a build and gives me a preview link to view the site with changes. Whenever I merge into the main branch, Netlify builds and deploys my site for me.

Linking the deploy to updating my README

Later in my journey, I wanted to learn about GitHub actions and built an action that pulls the three most recent blog blog down from my blog's RSS feed (included with Gatsby) and writes them to the markdown file in my profile README repo. You can read about the process of creating it in my past blog post.

A feature that I wanted was a connection between deploying my blog to prod and having the GitHub action scrape my RSS feed to update my README file. In the past, I had it running as a recurring Cron job, but I wanted something that would have a guarantee of the latest three blog blog being up to date. I had looked into using Netlify's outgoing webhook feature, but found it to be challenging to customize the POST request to match what was needed to trigger a GitHub action.

I realized that I always wait for the Netlify deploy to prod to complete, manually check the site, and delete the branch...so I decided to utilize the branch deletion action to trigger the scrape of the RSS feed to update my README repo. I then discovered that the GitHub action repository_dispatch would be helpful in triggering the scrape of my RSS feed on branch delete.

I first created a token to save as a secret in my danielle-heberling-dot-xyz (blog) repository that gave it read/write permissions to my deeheber (README) repository. I then added a workflow to my danielle-heberling-dot-xyz repo to post an event to my deeheber repo via repository_dispatch. It looks like this:

name: alert README repo of changes on main

on:
  workflow_dispatch:
  # Runs on git reference deletion (branch or tag)
  delete:

jobs:
  dispatch:
    runs-on: ubuntu-latest
    steps:
      - name: Update README in deeheber/deeheber repo
        run: |
          curl -H "Accept: application/vnd.github.everest-preview+json" \
          -H "Authorization: token ${{ secrets.DISPATCH_TOKEN }}" \
          --request POST \
          --data '{"event_type": "scrape", "client_payload": {"ref": "${{ github.ref }}"}' https://api.github.com/repos/deeheber/deeheber/dispatches

I noticed there were some pre-made actions that can do this for me, but I decided that since this is a simple POST request that I can do this via curl without the extra dependency of a 3rd party maintained GitHub action.

Then on the receiving repo (deeheber), I added an action to listen for that scrape event_type to trigger scraping my RSS feed and writing the three most recent blog blog/links to the README file. The on section is the only part that I had to change for this to work:

name: Build README

on:
  workflow_dispatch:
  repository_dispatch:
    types: [scrape]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Check out repo
        uses: actions/checkout@v3
      - name: Use Node
        uses: actions/setup-node@v3
        with:
          node-version: "18.x"
      - name: Install node dependencies
        run: npm install
      - name: Check for RSS feed updates
        run: npm run scrape
      - name: Commit and push
        run: |-
          git diff
          git config --global user.email "actions@users.noreply.github.com"
          git config --global user.name "README-bot"
          git add -A
          git commit -m "Update content" || git commit --allow-empty -m "Empty commit"
          git push

On both actions, I decided to keep the workflow_dispatch event in place, so I could manually run these workflows.

Closing

There are many different ways that I could've accomplished this, but this is what I ended up implementing. It is not perfect, but more than good enough for a personal blog site in my opinion.

Since I'm already using Netlify for hosting, I figured it was easiest to utilize their built in CI/CD services. I could've spent time rolling this myself using various AWS services or other things, but I wanted something that I didn't need to spend that much time on with the initial setup and/or maintence.

Thank you to Netlify, Gatsby, and GitHub for having a very permissive free tier that enables me to keep my blog up and running. I only need to pay for my custom domain name (danielleheberling.xyz) in order to keep things running.

Edit October 8, 2023: Sadly Netlify is amongst the many tech companies that laid off a lot of employees including folks on the Gatsby team. My thoughts are with those who lost their jobs. I hope they find new opportunities soon. I'm not sure what this means for the future of Netlify and Gatsby, but I hope they can continue to provide a great service to their customers. Time will tell.

Edit July 8, 2024 Update: The blog website mentioned in this post has been retired. It has been rebuilt using Astro and deployed to Cloudflare Pages.

Career Reflections

Danielle Heberling — Fri, 07 Jul 2023 22:12:03 GMT

Photo by <a href="https://unsplash.com/@claybanks?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Clay Banks</a> on <a href="https://unsplash.com/photos/LjqARJaJotc?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

Growing up, I was given advice from adults often. It was typically some variant of:

Go to college
Respect authority
Work hard
You'll be rewarded

Early in my career, I followed this advice and it didn't work for me. I could write an entire book on why that was the case, but for now I'll spare you the details.

On top of this bad advice, I found that in American culture the first thing people will often ask when meeting someone for the first time is what they do for a living. How you answer this question will result in judgement that may or may not be accurate.

As my career progressed, I realized that I aspire to have a legacy beyond the company that I work for and my job title. As a result, I came up with these five core principles. Here they are:

Be kind to people despite their position in the overall hierarchy. This includes both official heiarchy such as at a company and social heiarchy in a community setting. You never know when circumstances might change and when you might run into an individual again. I've mentored people who have exceeded my technical level of expertise as well as seen people who worked for competitors become my coworker.
Work in public. And don't limit what you publicly share to wins. Also share your challenges, so people can have a full picture of the work that went into the finished product. A rising tide lifts all boats.
Approach disagreement with a curious mind. Don't join a conversation expecting to "be right." Ask questions if someone else's perspective doesn't match yours. You might not end up agreeing in the end, but it's likely you'll learn something new.
Don't jump to technology to solve a problem unless it's the best tool for the job. I've worked many places where tech was thrown at problems in an attempt to solve them. This ended up causing more confusion while the root cause of the original problems were unaddressed. Sometimes problems can be resolved by training people or by updating a process.
Leave things better than you found them. Most individuals apspire to change the world, but it is challenging to do this at the individual level. At the very least, we can do our part by improving the current situation.

While thinking through these core principles, I've realized that people and community are why I'm still here in tech. To me, community looks like current and former coworkers, colleagues I met at community events, and people who use the things I build. I've gotten to where I am because people were willing to take a chance on me. I'd love to pay it forward by taking a chance on others and doing whatever I can to lift others up.

AWS Speakers Directory - Adding AI Functionality

Danielle Heberling — Thu, 29 Jun 2023 22:12:03 GMT

Photo by <a href="https://unsplash.com/@8moments?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Simon Berger</a> on <a href="https://unsplash.com/s/photos/rainbow-mountain?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

Background

In June 2023, Johannes Koch, Matt Morgan, Julian Michel, and myself participated in the AI themed hackathon for the AWS Community Builders. The hackathon challenge was to create a tool using the Transformer Tools framework.

The Project

Johannes had the idea that we agreed to implement. We ended up building an AWS community focused speaker directory website. Think an awesome AWS focused version of sessionize where event organizers can create an event. Speakers can create a speaker profile and use the site to submit talks to the various events. For more details on the project, take a look at Matt's overview post.

This post focuses on how we utilized the Hugging Face transformers agent to add functionality to our speaker directory.

Here's links to blog by my teammates on other topics related to the project

AI Functionality

The AI functionality that we added to enhance the speaker directory was

Generate an image whenever a speaker adds a new talk
Generate appropriate tags and add tags to newly added talks
Allow event organizers to get recommendations of talks that fit their event based off of a set of tags

Implementation Overview

Architecture

Features #1 and #2 were set this up in a similar way. We have a DynamoDB (DDB) event source that triggers both Functions via a DDB stream whenever a new talk is added via an AppSync GraphQL (GQL) mutation. The two Functions then do the work of generating images and dynamically adding tags to the talk.

The final step of the tag generation Function (feature #2) is to insert the generated tags into our DDB table. We then have another DDB stream that triggers when a tag is inserted that then triggers a Function that does some aggregation for us to support feature #3.

For feature #3, we set up a GQL mutation that accepts an array of tags and queries our DDB table to retrieve a list of talks (recommendations) that also contain those tags.

More on the Functions

Everyone on the team was new to the world of AI/ML, one thing we quickly realized was that these Functions were very heavy in terms of the size/number of dependencies needed and in terms of the compute power needed to complete these complex AI processing tasks.

Inspired by this blog post, we decided to utilize an EFS file system to cache some artifacts to lessen cold start on subsequent requests to these AI processor Functions. Due to the number of dependencies needed, we decided to take advantage of packing our Function code using Docker containers instead of .zip files to take advantage of the higher size limit.

Matt also learned that sometimes maintainers are not great at updating the dependencies in their Docker images, and he ended up creating a custom Docker image based off of the image used in that linked blog post with updated Python and gradio-tools (we needed it for image generation).

Function code - Generate Image

This Function responds to the DDB insert event when a new talk is added, calls the Hugging Face agent to generate an image based off of the description of the talk, and puts the generated image into an S3 bucket.

The new part for us was working with the Hugging Face agent. We first needed to login using credentials stored in Secrets Manager and generate the image using stable diffusion/gradio tools.

from gradio_tools import StableDiffusionPromptGeneratorTool
from huggingface_hub import login
import boto3
from transformers import HfAgent, Tool

gradio_tool = StableDiffusionPromptGeneratorTool()
tool = Tool.from_gradio(gradio_tool)
ssm = boto3.client("ssm")

agent_token = ssm.get_parameter(
    Name="/community-speakers/agent-token", WithDecryption=True
)
login(agent_token["Parameter"]["Value"])

agent = HfAgent(
    "https://api-inference.huggingface.co/models/bigcode/starcoder",
    additional_tools=[tool],
)

The entire Function code looked like this

import boto3

import os
from gradio_tools import StableDiffusionPromptGeneratorTool
from huggingface_hub import login
from io import BytesIO
from transformers import HfAgent, Tool
import json

BUCKET_NAME = os.getenv("BUCKET_NAME")
gradio_tool = StableDiffusionPromptGeneratorTool()
s3 = boto3.resource("s3")
ssm = boto3.client("ssm")
tool = Tool.from_gradio(gradio_tool)

agent_token = ssm.get_parameter(
    Name="/community-speakers/agent-token", WithDecryption=True
)
login(agent_token["Parameter"]["Value"])
agent = HfAgent(
    "https://api-inference.huggingface.co/models/bigcode/starcoder",
    additional_tools=[tool],
)


def handler(event, context):
    print(json.dumps(event))

    image = event["Records"][0]["dynamodb"]["NewImage"]

    image = agent.run(
        "Generate an image of the `prompt` after improving it.",
        remote=True,
        prompt=f"{image['description']['S']} {image['title']['S']}",
    )
    buffer = BytesIO()

    image.to_raw().save(buffer, "png")
    buffer.seek(0)

    prefix = "ai-generated"
    newId = (
        event["Records"][0]["dynamodb"]["Keys"]["pk"]["S"]
        .replace("#", "")
        .replace(" ", "")
    )
    s3.Bucket(BUCKET_NAME).put_object(Body=buffer, Key=f"{prefix}/{str(newId)}.png")
    response = {"statusCode": 200, "body": "ok"}
    return response

Function Code - Generate Tags

This function utilized a similar pattern in terms of connecting to Hugging Face; however, we utilized a different tools for this - the Bart model.

import boto3
import os
from datetime import datetime
from transformers import pipeline

oracle = pipeline(model="facebook/bart-large-mnli")

tableName = os.getenv("TABLE_NAME")

client = boto3.client("dynamodb")


def handler(event, context):
    tags = get_tags()

    candidate_labels = [t["tagName"]["S"].lower() for t in tags]
    image = event["Records"][0]["dynamodb"]["NewImage"]

    # Use both the title and the description of the talk to identify tags
    identifiedTags = oracle(
        f'{image["title"]["S"].lower()} {image["description"]["S"].lower()}',
        candidate_labels=candidate_labels,
    )

    labels = identifiedTags["labels"]
    scores = identifiedTags["scores"]

    selectedTags = [t for index, t in enumerate(labels) if scores[index] > 0.3]

    save_tags(image, selectedTags)

    response = {"statusCode": 200, "body": identifiedTags}
    return response


# Query all tags to get a list of label candidates.
def get_tags():
    response = client.query(
        ExpressionAttributeNames={"#pk": "pk"},
        ExpressionAttributeValues={":tag": {"S": "tag"}},
        KeyConditionExpression="#pk = :tag",
        TableName=tableName,
    )

    tags = response["Items"]

    # Need the loop to get all tags.
    while "LastEvaluatedKey" in response:
        response = client.query(
            ExclusiveStartKey=response["LastEvaluatedKey"],
            ExpressionAttributeNames={"#pk": "pk"},
            ExpressionAttributeValues={":tag": {"S": "tag"}},
            KeyConditionExpression="#pk = :tag",
            TableName=tableName,
        )
        tags.update(response["Items"])

    return tags


# We're going to save any tags that the creator may have neglected to apply.
def save_tags(image, tags):
    tagList = list(set([t["S"] for t in image["tags"]["L"]] + tags))
    print(tagList)

    # Update the original talk with any new tags
    client.update_item(
        ExpressionAttributeNames={
            "#tags": "tags",
            "#_md": "_md",
        },
        ExpressionAttributeValues={
            ":tags": {"L": [{"S": t} for t in tagList]},
            ":_md": {"S": datetime.now().isoformat()},
        },
        Key={
            "pk": {"S": image["pk"]["S"]},
            "sk": {"S": image["sk"]["S"]},
        },
        UpdateExpression="SET #tags = :tags, #_md = :_md",
        TableName=tableName,
    )

    # Save any new tags
    for tag in tags:
        try:
            client.put_item(
                ConditionExpression="attribute_not_exists(pk) and attribute_not_exists(sk)",
                Item={
                    "pk": {"S": image["pk"]["S"]},
                    "sk": {"S": f"tag#{tag}"},
                    "gsi1pk": {"S": f"tag#{tag}"},
                    "gsi1sk": {"S": image["pk"]["S"]},
                    "tagName": {"S": tag},
                    "title": {"S": image["title"]["S"]},
                    "_et": {"S": "TAG"},
                    "_ct": {"S": datetime.now().isoformat()},
                    "_md": {"S": datetime.now().isoformat()},
                },
                TableName=tableName,
            )
        except client.exceptions.ConditionalCheckFailedException:
            print("Tag already existed!")

Function Code - Aggregate Tags

This is triggered whenever a new tag is inserted into our table. As you'll see in the recommend talks Function, these aggregations support that feature. Because we don't need to utilize the Hugging Face transformers in this Function, we decided to do this one in Typescript rather than Python.

import { UpdateCommand } from "@aws-sdk/lib-dynamodb"
import { type DynamoDBStreamEvent } from "aws-lambda"

import { docClient, tableName } from "../util/docClient"

export const handler = async (event: DynamoDBStreamEvent): Promise<void> => {
  for (const record of event.Records) {
    const image = record.dynamodb?.NewImage
    if (!image?.tagName.S) continue

    const tagName = image.tagName.S

    const command = new UpdateCommand({
      ExpressionAttributeNames: {
        "#quantity": "quantity",
        "#tagName": "tagName",
        "#_et": "_et",
        "#_ct": "_ct",
        "#_md": "_md",
      },
      ExpressionAttributeValues: {
        ":one": 1,
        ":tagName": tagName,
        ":timestamp": new Date().toISOString(),
        ":zero": 0,
        ":_et": "TAG_COUNT",
      },
      Key: {
        pk: "tag",
        sk: `tag#${tagName}`,
      },
      TableName: tableName,
      UpdateExpression: `SET #quantity = :one + if_not_exists(#quantity, :zero),
                             #tagName = :tagName,
                             #_et = :_et,
                             #_ct = if_not_exists(#_ct, :timestamp),
                             #_md = :timestamp
                         `,
    })

    await docClient.send(command)
  }
}

Function Code - Recommend Talks

This Function is set up as a GQL resolver that is called from our Flutter frontend. We decided to set this one up as a Lambda resolver rather than a JS pipeline resolver because this feature is making multiple queries to our table. This Function also does not utilize the Hugging Face transformers, so we decided to write it using Typescript rather than Python.

import { QueryCommand } from "@aws-sdk/lib-dynamodb"
import { type AppSyncResolverEvent } from "aws-lambda"

import { type Event, type Talk } from "../API"
import { docClient, tableName } from "../util/docClient"

export const handler = async (
  event: AppSyncResolverEvent<{ event: Event }>,
): Promise<Talk[]> => {
  const eventTags = event.arguments.event.tags
  const talks: Array<Talk & { matches?: number }> = []
  for (const eventTag of eventTags) {
    const queryTagsCommand = new QueryCommand({
      ExpressionAttributeNames: {
        "#gsi1pk": "gsi1pk",
      },
      ExpressionAttributeValues: {
        ":gsi1pk": `tag#${eventTag}`,
      },
      KeyConditionExpression: "#gsi1pk = :gsi1pk",
      IndexName: "gsi1",
      TableName: tableName,
    })
    const tagsResult = await docClient.send(queryTagsCommand)
    if (!tagsResult.Items) continue

    for (const tagResult of tagsResult.Items) {
      const queryTalksCommand = new QueryCommand({
        ExpressionAttributeNames: {
          "#pk": "pk",
          "#sk": "sk",
        },
        ExpressionAttributeValues: {
          ":pk": tagResult.gsi1sk,
          ":talk": "talk",
        },
        KeyConditionExpression: "#pk = :pk and begins_with(#sk, :talk)",
        TableName: tableName,
      })
      const talksResult = await docClient.send(queryTalksCommand)
      if (talksResult.Items) {
        talksResult.Items.forEach((talk) => {
          if (!talks.find((t) => t.pk === talk.pk)) {
            talks.push(talk as Talk)
          }
        })
      }
    }
  }

  talks.forEach((talk) => {
    const matchCount = (talk.tags || []).filter((t) =>
      eventTags.includes(t),
    ).length
    talk.matches = matchCount
  })

  return talks
    .filter((t) => t.matches)
    .sort((a, b) => (a.matches && b.matches && a.matches < b.matches ? 1 : -1))
    .slice(0, 10)
}

Trade offs

We noticed that generative AI takes a lot longer to process than your typical CRUD-like API against a database. For this reason a lot of our processing was conducted in an async manner. Not necessarily a trade off, but we did need to change our mental model in how we approached this as it was not going to be a good user experience if we stuck with the more traditional request/response model.

An issue that we ran into was that the shared AWS account we were using for the hackathon was considered "new" and had some additional usage limits on it. In our case, the Lambda Functions had an upper limit of 3008 mb. Most of the time, this is more than enough memory for a request/response API endpoint that interfaces with a database; however, with AI/ML having more memory would've helped with our slow responses during the processing.

We wrote a ticket to AWS support and were told that these limits are in place on newer accounts to prevent surprise bills. Then they told us they couldn't raise the limit until they saw more activity in the account. I can't speak for my teammates, but it is my personal hope that AWS can improve this process and live up to the "customer obsession" leadership principle. We ran into other issues with the "new" account limits, but that is the only one I'll mention here since it is directly related to our AI integrations.

Post Hackathon Plans

Given the timeline for this hackathon (30 days) and the fact that no one on this team had extensive experience with AI/ML, we took some shortcuts when it came to the talk recommendations feature. Overall, I'm happy with how it came out. Post hackathon, we plan to explore creating and training our own custom data model based off of what was entered into our DDB table in order to see if we can further lean on AI/ML and get more specific (beyond tags) recommendations.

I would've liked to go beyond the Hugging Face transformers and see if some of this could be implemented using SageMaker endpoints or another AI platform. Potential benefits that I saw with SageMaker endpoints over our implementation was that this is in the AWS ecosystem for a (hopefully) tighter integration and being able to speed up processing. I learned from this hackathon that Hugging Face is great, but there might also be other tools out there that better suit our needs. We won't know for sure until we try!

Closing

Overall this was a fun collaboration. It feels good to contribute to building something that the community will use and to learn from my peers within the community while doing so.

On a personal level, life got in the way and I did not have as much time as I would've liked to have to dedicate to this project. I thank my team mates for their hard work via ideas and implementation. Major props to Matt for doing a lot of the AI implementation. 🙌🏻

If you would like to participate with us building out the AWS Speaker Directory to help AWS User Group leaders find speakers, please reach out of any of us.

The Problem with "It Works on My Computer"

Danielle Heberling — Mon, 20 Mar 2023 10:12:03 GMT

Disclaimer: This is an opinionated post based off of my own experience. Your experience and opinions may differ and that is okay. I welcome your respectful and engaged conversation. Most of my cloud experience is with Amazon Web Services, so this post will be AWS heavy. A lot of this applicable to other cloud providers though.

Scenario #1: Working on a serverless first team

We hire a new person that's new to serverless. Regardless of how long they've been a professional software developer, the first question is often some variation of:

"How do I run this locally on my computer?"

There are many tools out there that can emulate AWS on your computer, but I've experienced some issues with this approach.

The tools are not always accurate or up to date with the latest features available in the cloud.
The environment on your computer vs in the cloud will be different. Some common examples include: environment variables, database connections, and permissions to access resources.
Maintaining the tooling required to emulate can sometimes be more work than deploying your changes and testing in the cloud. A lot of them require knowledge of Docker and Docker networking. My biased opinion: at that point why go serverless in the first place?

Scenario #2: Working on a non-serverless first team

Some companies use more traditional infrastructure such as EC2 instances and RDS. Running a local version of a Node.js server against a local Postgres database is great for fast, iterative development and works well most of the time.

Where this falls apart is when integrating another AWS service such as S3. I've seen software developers who do not have much cloud experience, but are great at writing code do the following:

Authenticate with AWS using administrative access credentials locally on their computer.
Run a server locally on their computer which uploads a file to an S3 bucket located in a non-prod AWS account.
It works!
They submit their code and it gets merged into the main branch.
The code gets deployed to a non-prod environment (staging or something similar).
They test uploading to the S3 bucket and it fails on the non-prod environment.
"I don't know what's going on because it worked locally for me!" 🤔

Photo by <a href="https://unsplash.com/@sammywilliams?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Sander Sammy</a> on <a href="https://unsplash.com/photos/ufgOEVZuHgM?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

Someone more experienced with IAM recognizes that the permissions on the cloud and the local environment differ.
The more experienced person fixes the permissions issues in the cloud. 🚀

Unsolicited advice

Have a non-prod cloud environment where you can test code changes before deploying to prod. It's good to catch problems before your end users do. ❤️
If you're running servers locally as part of your development loop, authenticate locally using similar permissions that the production server uses. This will help catch permissions issues sooner. 🤓
If you're working on a serverless first team, do not attempt to run your application locally. Instead, deploy your code to a sandbox account and test it there. One AWS account per developer. It's also a great way to get more familiar with troubleshooting in the cloud. 🙌🏻

Reflections

I've seen these scenarios play out on many different teams at different companies. The reason is due to developers needing to change their mindset.

Today it is very common for software applications to use cloud services. We as developers should also evolve our development environments to match the new paradigm.

Let's build better software together. Good luck out there. 💪🏻

Serverless Weather Reporting with AWS Step Functions and CDK

Danielle Heberling — Sun, 05 Mar 2023 10:12:03 GMT

Photo by <a href="https://unsplash.com/@devjustesen?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Devin Justesen</a> on <a href="https://unsplash.com/photos/QrL-aRyuf_8?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

Background

I live in the Pacific Northwest. Approximately 1-2 times per year, it snows. When this happens, people act shocked because "it never happens here."* Our local governments do not have many snow plows nor do they salt the roads much. This means that when it snows, the roads are highly dangerous. The city of Portland, Oregon often shuts down for multiple days.

Former residents of Portland created a website, isitsnowinginpdx.com. I cannot speak for their motivations, but I find this site both entertaining and informative.

After moving out of Portland into the suburbs, I noticed that the site was not as accurate, so I decided to build my own site isitsnowinginhillsboro.com.

Architecture Overview

Behind the scenes, this site is using AWS Serverless technologies. S3 hosts site assets, a Step Function updates the site, and EventBridge Scheduler triggers the Step Function every 10 minutes.

I made the decision to not put CloudFront in front of this S3 Bucket, because this is a region specific website. Most people viewing this site live in the area and adding a CDN feels like unnecessary complexity given the use case. As a result, the site renders of HTTP rather than HTTPS.

Here's a breakdown of what the Step Function workflow does:

Get the site's current status (snow or no snow) from a DynamoDB table
Check the current weather using the OpenWeatherMap API via a Lambda Function
A choice state compares the two values
If the two values match, nothing happens and the workflow ends
If the two values do not match, it updates the site to show the current weather obtained from the API
- A Lambda Function generates the HTML and places that HTML file into an S3 bucket (more on this later)
- If the Lambda Function is successful, then it updates the site status in DynamoDB

Here's a visual representation of the workflow:

Challenges

Overall things went smoothly. I did run into a few tiny issues.

Issue 1: EventBridge Scheduler and CDK

EventBridge Scheduler is a new-ish feature and does not have an L2 construct. I was able to implement what I needed for this site using an L1 construct, but with an L1 construct you're writing Cloudformation without the benefits that an L2 construct provides. In this case, I had to define an IAM Role to allow the Scheduler to invoke the Step Function and an inline policy. This is likely a feature that would be included in an L2 construct.

More info on L1 vs L2 constructs here. I plan to keep an eye on this open RFC.

Issue 2: HTML Generation

Step Functions have direct integrations with many services. The advantage of this is you do not have write a Lambda Function to make those AWS SDK calls and can save money as well as enjoy the built in error/retry logic that comes with the Step Function service. I used the direct integration for the two calls to DynamoDB and it worked quite well.

When it came to generating HTML and using the S3 PutObject, the direct integration added quotes around the HTML. This resulted in an HTML document that looked similar to "<h1>My site<h1>". The extra quotation marks caused the page to not render properly in the browser. I eventually got this working with a Lambda Function by sending the Body as a Buffer, but I prefer the direct integration.

Your Turn

The code is open source and you can view it here.

You can clone the repo following the instructions in the README.md file to create your own weather site. Plug in the latitude and longitude of your city. Also, it's not limited to snow! Check out the Main weather types here for all available options.

Open source contributions are welcome.

Thanks to the original creators of isitsnowinginpdx.com for the inspiration.

* To be fair the last instance of snow in the area was more than usual.

** Edit February 23, 2024: I've added CloudFront to this. It's nice to have the SSL cert, but to me the work felt like overkill in terms of effort and the extra infra needed for SSL. Hopefully AWS can improve this experience.

Improved git workflows with graphite.dev

Danielle Heberling — Sun, 02 Oct 2022 09:12:03 GMT

Photo by <a href="https://unsplash.com/@richygreat?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Richy Great</a> on <a href="https://unsplash.com/s/photos/github?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

Current Landscape

Git is a tool that many tech people use. It's great for the most part. There are some pain points that could improve the experience. My issues revolve around collaboration with others. Here's some examples.

As someone requesting a code review, so I can merge my changes:

waiting for someone to review my code
merge conflicts due to a PR being open for very long
sometimes the code in the PR is needed for next steps

As someone doing code reviews for my teammates:

it can be unclear which PRs I need to review or re-review
navigating very long PRs

Solution

Upon joining my most recent company, my onboarding buddy told me about how they use graphite.

Graphite utilizes something they're calling a "stack." What used to be one huge PR is now many branches that build off of each other with each branch being its own PR.

Sounds great as someone who has to review code, but as the code submitter I had some concerns about what happens when merging branches. Would I get lots of merge conflicts? So far while using graphite, the answer has been mostly no. Graphite has a cli command that pulls down changes from the upstream and automatically rebases all affected branches.

gt repo sync --restack

A nice bonus is that this command also prompts to ask if you want to delete local branches that have been merged into the main branch.

Another benefit is that graphite has a nice web based UI where you can see which PRs need your input both as a PR submitter and reviewer.

Summary

Despite all of the custom cli commands, graphite uses git under the hood. If you want to do something with vanilla git, graphite will pass that command through to git directly. One con to using graphite is that it just appears to work with Github (for now?). Also, at the time of writing this, there's a waitlist to get on the app.

There's many more cli commands and features that I did not cover. Much like git, people have their preferred workflows and there's many more commands/options than one would use in their daily workflow.

Disclaimer that I was not paid nor asked to write this post. I personally found this tool helpful. Thought I'd share about how it's helped me with the intent that it might also help you.

AWS Application Composer, the App Building Future We Need

Danielle Heberling — Wed, 21 Sep 2022 05:17:19 GMT

At re:Invent 2022, Werner Vogels announced the preview of <a href="https://aws.amazon.com/application-composer/" target="_blank" rel="noopener noreferrer">AWS Application Composer</a>.

I don't want to write an overview or "getting started" guide, because you can read <a href="https://dev.to/aws-builders/overview-of-aws-application-composer-3j34" target="_blank" rel="noopener noreferrer">fantastic ones from people in the community</a>. The focus of this post will be an overview of the problems that I see this service solving and what I'd want for the future.

The Problem

A promise of serverless is being able focus on business logic instead of infrastructure. While I agree with this goal, I don't think it has been fully achieved yet in practice.

Unless you're someone who has a background in operations or cloud, it can be difficult to get started. Once you get started, it can be challenging to move fast...let alone move fast with a team.

A best practice is to utilize infrastructure as code (IaC) to manage and provision cloud resources, because this allows you to have a written/deployable representation of your resources that can be shared in version control with a team. This is wonderful, but if you don't know anything about the resources you're deploying it can be difficult to find a starting point when authoring these files.

Not only is writing the IaC for individual resources challenging, so is figuring out how to connect multiple resources together for a real world application. Things like IAM policies, event triggers for resources such as Lambda, and being able to add environment variables to resources that reference other resources in your stack come to mind.

The Solution (so far)

Since the beginning of my cloud/serverless journey, there has always been a high demand for a visual way to drag/drop resources that generate deployable IaC. I've seen individuals in the community and companies attempt to build a solution, but none have resulted in widespread adoption.

This problem space is what drew me to work as a software engineer at Stackery. AWS's acquisition of Stackery is what began the journey to what would result in AWS Application Composer. I left the company shortly before the acquisition; however, I still believe strongly in the mission.

Credit: <a href="https://twitter.com/annaspies" target="_blank" rel="noopener noreferrer">Anna Spysz</a>

Features I'd like to see post GA

Much like the aspiration of being able to focus on business logic instead of infrastructure, I'd love to see a future where we don't have to write any IaC. This way, we can focus on architecture instead of resource configuration. It also has the potential to enable non-engineering participation in application building. While we're far away from that goal, I think we're getting closer with AWS Application Composer.

A popular criticism of AWS Application Composer has been that it outputs <a href="https://aws.amazon.com/serverless/sam/" target="_blank" rel="noopener noreferrer">AWS SAM</a> and not ${preferred IaC}. While those are valid requests (Terraform support would be awesome for my day job), I think the folks focusing on this (many of whom I respect) are missing the bigger picture.

With that, here's a bulleted list of other things I'd like to see in the future:

A way to use this outside of the AWS console - Some organizations restrict access to the console for regulatory reasons. Something like an IDE extension might help.
Amazon VPC resource - RDS requires a VPC.
AWS AppSync resource - GraphQL is amazing for mobile applications (and other use cases).
AWS Step Function resource integration into Workflow Studio - Allows for easier usage of a service that is often recommended by AWS employees to use in serverless applications.
One button push deployment from the console - It's great as is with <a href="https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/accelerate.html" target="_blank" rel="noopener noreferrer"> AWS SAM accelerate</a>...but I'd love to see it be easier.

If you have any suggestions, I recommend using the "submit feedback" link in the console with details as to why you want x feature. The team wants to hear your feedback and is actively using it to improve the product.

Closing thoughts

I think AWS's acquisition of Stackery makes a lot of sense and hopefully being directly embedded as a service within AWS will make it even better.

This team deeply cares about developer experience and reducing the friction that exists within AWS serverless. They are some of the most brilliant and humble folks I've ever had the pleasure to call coworkers. The team members whom I've interacted with who aren't former coworkers have been nothing but kind and helpful.

I hope that the dysfunction that sometimes exists with individual egos in big name corporations does not mess up the larger vision that started at Stackery. For now, I have faith that AWS can give the team the resources that they need to scale out this vision further. I very much look forward to what the future holds.

If you haven't yet, I encourage you to give it a try and provide feedback. It's still in preview and far from finished, but I think you'll be pleasantly surprised.

Making the Switch - Software Development on Pop!_OS

Danielle Heberling — Thu, 04 Aug 2022 10:12:03 GMT

My Background

Disclaimer: this post details my experiences using Pop!_OS for software development. Depending on your needs, it's possible your experience might differ. For this reason, here's the tools that I generally use for software development:

Node.js/JavaScript/TypeScript
AWS Serverless services
AWS SAM
AWS CDK

Why Switch Away from macOS?

Apple does a wonderful job of controlling the entire ecosystem for its devices. This is can be good since things "just work." But, if you want to go beyond that and want more flexibility it might not be the best choice.

I was finding myself wanting to be able to upgrade my computer's RAM quite a few times. Many Apple devices solder it to the logic board. You then have the option of 1. Buying a new logic board (almost the cost of a new computer) or 2. Buying a new computer. I didn't like these choices and decided to check out alternatives.

I should also mention that I worked for Apple. I feel conflicted supporting a corporation that did not treat me well during my time as an employee. No judgements on those who do choose to use Apple products since everyone has their reasons.

What is Pop!_OS?

The last time I used a Windows machine was Windows XP. A lot has changed since then, and I was hesitant to go back since I never enjoyed Windows anyway.

This is what gave me the idea to check out Linux. It's open source and there's many different distros one can choose. Plus, I was already familiar with the command line.

I settled on Pop!_OS because:

it's based on Ubuntu, a very common distro that has a lot of documentation and an active community
it had a very similar UI to macOS - what I'm already familiar with
I could use apt-get to install or I could use the Pop!_Shop that comes with this distro (like the Mac App store)

My Setup

Here's what I installed on my new Pop!_OS machine for software development:

nvm - a tool to install and switch to different versions of node
Visual Studio Code - a code editor that I downloaded from the Pop shop
AWS CLI
SAM CLI - the docs suggest using Homebrew, but the idea of using Homebrew on a Linux machine feels wrong. I downloaded the file and ran the install that way. Eric Johnson, an AWS employee provided this gist for install scripts on Linux as an option.
Docker - I followed the instructions to install Docker Engine on Ubuntu; but, it looks like there's a Docker Desktop for Linux in GA now. 🎉 Also of note, docker compose performs way better on Linux machines (if you use that feature).

Conclusion

If you use similar tools that I do, you'll notice that not much has changed to install things. You have to follow the Linux (sometimes Ubuntu) specific directions.

If you would also like to get out of Apple's "walled garden," I'd recommend trying Pop!_OS. I've been using it as my daily driver now for about a year and have not had many issues. I'll admit, I was initially very hesitant since I was familiar with macOS...but this has been a great upgrade in my life.

Perhaps next year will be the year of Linux on the desktop. 😝

Are You Paying Attention?

Danielle Heberling — Sun, 13 Mar 2022 22:12:03 GMT

Every time I go out in public, I've noticed a trend. People are on their phones almost everywhere I look. We hike to a beautiful waterfall, only to find a crowd of people standing there recording it on their phones for whatever social media site is popular. I go to a live show and the phones are out recording.

I've noticed this change within myself as well. I'm sitting in my living room and constantly feel compelled to look at my phone. Sometimes when I can't focus at work, I just open up Twitter to the endless scroll. At the end of the day, I find behaviors like this are making me feel less happy in life and more stressed/angry, so I've been actively trying to reduce this behavior.

Apps Are Designed to be Addictive

The endless scrolling and constant notifications are designed to make users open up the app and engage more. As a software engineer, I've been in conversations that sounded something like "well how can we design this to encourage users to [insert action that makes the company money here]." I get that it's a business and there's a need to make money, but this makes me feel gross. People are constantly bombarded with apps and notifications trying to control their attention, and I would rather not be contributing to that. I much prefer the question to be phrased something like "how can we encourage users to love using our app" or "how can we make it easier for people to use our app."

Free social media sites in particular are probably the worst offenders. The model of how they make money is through advertisers, so they're incentivized to keep you on their site as long as they can in order to collect as much data about you to share with advertisers. They're also hoping your eyes read the ads to prompt you to purchase something you normally wouldn't. In this case you're truly "paying" with your attention.

Outrage and arguing drives the most engagement, so many of these sites will prioritize blog in your feed that will outrage you into engaging. On top of that, everything just feels dehumanized because we're communicating in text and can't actually see the person we are communicating with. Excluding the bots, there are humans on the other side of these blog. It's easy to forget that.

As far as I'm concerned, I'd love to see surveilance capitalism end because it is ruining many people's ability to pay attention to what's really important in life. The first step is realizing what is going on, so we can work together to create meaningful change in society.

Coping Mechanisms I've been Using

In the meantime, I still need to live in this world the way it is. Here's some techniques I've used to help get some of my ability to focus on things that matter back. Perhaps they'll also help you?

Take a break from computer screens

It's super easy to start scrolling and just lose track of time. I really like the pomodoro timer app flow to remind me it's time to take a break.

Go on walks

Getting outside is great, but it's also good to do this without a phone. Just let your mind wander as you walk around and it'll help you to unwind as well as come up with some shower thought ideas.

Meditation

Taking some time out of the day to just focus on breathing has been helpful for me. It calms me down and helps to center my mind. There's lots of ways you can do this. I personally enjoy using headspace.

Read books and physically write down notes

The medium in which you consume content influences how you consume it. For example social media blog are generally short blips of thoughts. So your mind spends time hopping around from thought to thought. I've found reading entire books to be helpful. A book is one cohesive thought and they're generally free of distractions such as advertising.

Whenever I need to take notes, I find physically writing things down helps me to retain the information. If I type it out on my computer, I find myself opening a second tab and navigating to various websites.

For reading and note taking I use a Boox Nova 3 with very few apps installed on it, because I don't like having many pieces of paper and notebooks laying around.

Chase flow states

Have you ever started working on a task and just had so much fun with it that time stood still? That's a flow state. For me a good flow state is sitting down in front of the piano and practicing some songs. Whenever I get caught up in the scroll of social media, I sit and think to myself "what could I be doing now that would be more rewarding" and then I go do it. Chase those flow states for maximum happiness in life.

Closing

If you're like me and are having troubles focusing on things that matter in your life, know that you're not alone and it's not completely your fault. There are small things that you can do; however, we need a big societal change as well.

This post was inspired from my life experience as well as from reading Stolen Focus. If you're interested in learning more, this book is a great place to start. All of the products I mentioned in this post are things that have helped me in my life, and I am not getting compensated to endorse.

Upgrading to CDK v2 for Typescript

Danielle Heberling — Tue, 18 Jan 2022 22:12:03 GMT

As of re:Invent 2021, CDK v2 is <a href="https://aws.amazon.com/about-aws/whats-new/2021/12/aws-cloud-development-kit-cdk-generally-available/" target="_blank" rel="noopener noreferrer">now generally available</a>. 🎉

This post includes a brief walkthrough on upgrading a project from v1 -> v2 as well as some personal opinions. I'll be using <a href="https://www.danielleheberling.xyz/blog/appsync-cdk/" target="_blank" rel="noopener noreferrer">this notes app</a> that we build previously to demo. If you want to skip ahead, the finished source code is <a href="https://github.com/deeheber/note-service-next-generation/tree/blog-post-2" target="_blank" rel="noopener noreferrer">on Github</a>.

Here's we go. Let's upgrade the application!

Step 1: Disable feature flags in cdk.json

Many of the feature flags from v1 are now included in v2 and/or are no longer relevant, so let's disable all flags.

Before

{
  "app": "npx ts-node --prefer-ts-exts bin/note-service.ts",
  "context": {
    "@aws-cdk/core:enableStackNameDuplicates": "true",
    "aws-cdk:enableDiffNoFail": "true",
    "@aws-cdk/core:stackRelativeExports": "true",
    "@aws-cdk/aws-ecr-assets:dockerIgnoreSupport": true,
    "@aws-cdk/aws-secretsmanager:parseOwnedSecretName": true,
    "@aws-cdk/aws-kms:defaultKeyPolicies": true,
    "@aws-cdk/aws-s3:grantWriteWithoutAcl": true
  }
}

After

{
  "app": "npx ts-node --prefer-ts-exts bin/note-service.ts",
  "context": {
    "@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId": false,
    "@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021": false,
    "@aws-cdk/aws-rds:lowercaseDbIdentifier": false,
    "@aws-cdk/core:stackRelativeExports": false
  }
}

Step 2: Update package.json dependencies

One of the features of CDK v2 is having mostly everything in a single library, aws-cdk-lib. Hopefully this can help alleviate some of the pain that developers experienced with keeping all packages up to date and in sync.

Let's focus on the devDependencies section of the package.json. In v2, we also need to add a peerDependencies section.

Before

  "devDependencies": {
    "@aws-cdk/assert": "^1.139.0",
    "@aws-cdk/aws-appsync": "^1.139.0",
    "@aws-cdk/aws-dynamodb": "^1.139.0",
    "@aws-cdk/aws-lambda": "^1.139.0",
    "@aws-cdk/aws-lambda-nodejs": "^1.139.0",
    "@aws-cdk/aws-logs": "^1.139.0",
    "@aws-cdk/core": "^1.139.0",
    "@types/jest": "^27.4.0",
    "@types/node": "^17.0.8",
    "aws-cdk": "^1.139.0",
    "esbuild": "^0.14.11",
    "jest": "^27.4.7",
    "ts-jest": "^27.0.2",
    "ts-node": "^10.4.0",
    "typescript": "^4.5.4"
  },

After

  "peerDependencies": {
    "aws-cdk-lib": "^2.8.0",
    "constructs": "^10.0.36"
  },
  "devDependencies": {
    "@aws-cdk/assert": "^2.8.0",
    "@aws-cdk/aws-appsync-alpha": "^2.8.0-alpha.0",
    "@types/jest": "^27.4.0",
    "@types/node": "^17.0.8",
    "aws-cdk-lib": "^2.8.0",
    "constructs": "^10.0.36",
    "esbuild": "^0.14.11",
    "jest": "^27.4.7",
    "ts-jest": "^27.0.2",
    "ts-node": "^10.4.0",
    "typescript": "^4.5.4"
  },

One thing to note in this specific project is that the AppSync dependency is currently in alpha. Personally, I would recommend exercising extreme caution before using it in a business critical application.

Step 3: Update import statements in the code

Since we've changed some dependencies in our package.json, we should update our import statements to match.

Here's an example of what that looks like

Before

import {
  CfnOutput,
  Construct,
  RemovalPolicy,
  Stack,
  StackProps,
} from "@aws-cdk/core"
import { FieldLogLevel, GraphqlApi, Schema } from "@aws-cdk/aws-appsync"
import { AttributeType, BillingMode, Table } from "@aws-cdk/aws-dynamodb"
import { Architecture, Runtime } from "@aws-cdk/aws-lambda"
import { NodejsFunction } from "@aws-cdk/aws-lambda-nodejs"
import { RetentionDays } from "@aws-cdk/aws-logs"

After

import { Construct } from "constructs"
import { Stack, StackProps } from "aws-cdk-lib"
import { CfnOutput, RemovalPolicy } from "aws-cdk-lib"
import { AttributeType, BillingMode, Table } from "aws-cdk-lib/aws-dynamodb"
import { Architecture, Runtime } from "aws-cdk-lib/aws-lambda"
import { NodejsFunction } from "aws-cdk-lib/aws-lambda-nodejs"
import { RetentionDays } from "aws-cdk-lib/aws-logs"
import { FieldLogLevel, GraphqlApi, Schema } from "@aws-cdk/aws-appsync-alpha"

Step 4: Bootstrap your environment for v2

If you've previously been working with v1, you'll need to re-bootstrap your environment so you can deploy using CDK v2.

To do this run the following in your console after you've installed CDK v2 on your machine.

cdk bootstrap aws://<your-account-number>/<your-region>

Closing thoughts

From my end, the upgrade process was seamless. Keep in mind that this was just a small personal project not using many constructs, so your experience might possibly differ. Overall, the team at AWS did a great job writing documentation on this topic.

Despite the announcement of CDK v2 moving out of developer preview, I did notice that many constructs for v2 are still in alpha...so that seemed somewhat misleading. I'm excited about CDK v2, but I plan to wait a bit before upgrading business critical applications to ensure that constructs will work reliably with the new version. I am pretty excited about all of the improvements under the hood and look forward to using it more extensively in the future.

My Top Three AWS re:Invent 2021 Announcements

Danielle Heberling — Thu, 09 Dec 2021 22:12:03 GMT

Disclaimer that this post contains my individual opinions and yours might differ. Let's get started!

1. SQS Partial Batch Retries in Lambda

Photo by <a href="https://unsplash.com/@farber?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Jonathan Farber</a> on <a href="https://unsplash.com/s/photos/batch?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

What this means

This feature was announced during pre:Invent, but I think it still counts. Before, if SQS messages were sent as a batch into Lambda, you had the choice to return a response of a full failure or full success for the batch. Now we have an extra setting that allows queue retry behavior to be at the record level instead of the batch level. If one or more messages within that batch fail, you have the option to only send the failed messages back to the queue to attempt a retry.

Why it's exciting

This allows me to delete some code that I wrote to ensure each message is idempotent and also allows me to reduce the number of API calls made to check if a message was processed before.

2. AWS Cloud Development Kit (CDK) v2 is Generally Available

Photo by <a href="https://unsplash.com/@imattsmart?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">iMattSmart</a> on <a href="https://unsplash.com/s/photos/hammer?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

What this means

CDK v2 has been in developer preview for a while and it brings in some exciting features.

Notable changes include:

One dependency library for all AWS services
Stable APIs are enforced via semver
Introduction of cdk watch which auto-deploys code for you as you edit locally and save the changes. (Note that AWS does not recommend the use of this feature in production environments.)

Why it's exciting

AWS took customer feedback and addressed some of the common painpoints with the original release of CDK. To me, these changes show maturity in the tool and increases my confidence in using it for production workloads.

3. Sustainability Pillar for AWS Well-Architected Framework

Photo by <a href="https://unsplash.com/@karsten_wuerth?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Karsten Würth (➡️ @karsten.wuerth)</a> on <a href="https://unsplash.com/s/photos/environment?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

What this means

AWS released a whitepaper that outlines best practices and recommendations around how to run workloads in the cloud with a focus on sustainability.

Why it's exciting

Just because things are running in someone else's data center, doesn't mean that it isn't using up natural resources. With the recent unprecedented weather events, it's important to get started thinking about this now. The fact that AWS made this an official pillar shows that they also care about this.

Closing

It was difficult to choose just three, because there were many great announcements during this conference. Overall, the announcements felt more incremental than innovative which I totally support. It's much better to improve on an existing service than to continually launch new services that aren't fully ready for production.

I encourage you to check out all of the announcements here. What were some of your favorite announcements and why?

"Now go build" - Werner Vogels

On Home and Remote Work

Danielle Heberling — Tue, 23 Nov 2021 22:12:03 GMT

Photo by <a href="https://unsplash.com/@cutnshoot?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Jonathan Rivera</a> on <a href="https://unsplash.com/s/photos/western-pennsylvania?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

I spent the first 22 years of my life in a part of the United States that many would call "flyover country." To most people this is an insignificant small town that peaked in the 1950s. As my father likes to say "This town was so much better when the mafia ran everything."

To me this town is a lot more... to me it is

Where I was born and grew up
Where my family is
Where I first learned what community felt like
Where I learned that I loved performing music
Where I became the head drum major of my high school marching band

...in short, this town was home to me...

After high school graduation, I attended a state school in the area to earn a Bachelor Degree in Music Education. Taking a look around me, I noticed that teachers in local schools stayed in their positions for a long time. Upon graduation from university, I hit a crossroads where I had to choose between:

Substitute teaching for 10+ years until a teacher left their position
Moving somewhere else, getting some experience as a teacher, and moving back

Photo by <a href="https://unsplash.com/@ivalex?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Ivan Aleksic</a> on <a href="https://unsplash.com/s/photos/crossroads?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

I chose option #2 in the end. Through many twists and turns, I am no longer a music teacher, nor have I moved back. Nowhere that I've lived since the initial move has felt like home.

Now we are in the midst of the Covid-19 global pandemic where many workers are forced to work remotely, some for the first time. Many companies have stated that they're planning on keeping employees fully remote for the forseeable future.

I take a look at my young neice whose current bedroom is my childhood bedroom. The future of remote work gives me hope. Hope that my neice won't be forced into the same decision crossroad that I was. If she wants to move away, she still can...but with the increase of remote work it is less likely to be a decision that could impact how much money she can make or her career opportunities.

I don't regret the people and life experiences that I've had as a result of my choice. I do still wonder how much I've missed and ponder the possibility of an eventual return.

Afterall when we're gone, it doesn't matter what our rank was at our workplaces. It's the love we shared with the people around us. In the meantime, my neighbors will have to deal with my accent that slips out on ocassion.

Event Driven Background Processes

Danielle Heberling — Sun, 12 Sep 2021 07:12:03 GMT

One theme when talking about serverless computing that frequently comes up is that "serverless is event driven." Personally I've found this to be true and am going to share one of my favorite event driven architectures built with AWS in this post.

Business Scenario

For this post, we're going to use a simplified example application.

In our example application, whenever a user signs up to use our application, we want to perform the following three actions:

Count the event in our <a href="https://heap.io/" target="_blank" rel="noopener noreferrer">3rd party analytics service</a>
Put them on an email mailing list in our <a href="https://mailchimp.com/" target="_blank" rel="noopener noreferrer">Mailchimp account</a>
Send a notification to a <a href="https://slack.com/" target="_blank" rel="noopener noreferrer">Slack</a> channel in our corporate slack

Implementation

There's many ways someone could approach this issue, but my favorite way is to use a "fan out" or "event fork" approach. Here's how it works.

In our example use case, when the person signs up to use the application, we send a message to <a href="https://aws.amazon.com/sns/" target="_blank" rel="noopener noreferrer">Amazon SNS</a>. That SNS topic then receives that message and sends it to three <a href="https://aws.amazon.com/sqs/" target="_blank" rel="noopener noreferrer">queues</a>, each having a <a href="https://aws.amazon.com/lambda/" target="_blank" rel="noopener noreferrer">function</a> on the other side. Each function contains code that does the heavy lifting for us.

Message Filtering

In a real life application, you'll most likely have many many events that you want to track and perform actions as a result.

Let's revisit our example and add a new event we want to track - login. For login, we only want to count the event in our 3rd party analytics service and ignore adding them to a mailing list (they're already signed up) and ignore receiving a slack notification (that would be a lot of noise in slack).

We can accomplish this by utilizing <a href="https://docs.aws.amazon.com/sns/latest/dg/sns-message-filtering.html" target="_blank" rel="noopener noreferrer">SNS message filtering</a>.

Benefits to this Approach

Like any architecture decision, there are always pros and cons. The important thing is to weigh the two together and use what works for you and your team. Here's some benefits I've noticed and why I enjoy using this approach:

The events are decoupled from the initial action. For example, we wouldn't want someone to not be able to sign up just because there was an error counting the signup in our analytics.
These background reactions to events don't need to be instant since we don't need to provide the user feedback...it's just for our backend office uses, so it is fine to process these events seperate from the general application flow.
Let's say you have a CLI tool and a web application...both of those can send messages to this SNS topic.
If you decide you no longer want to use Mailchimp as your email list provider, you can delete the corresponding queue/function combo and add a new queue/function to process your replacement mailing list service.

Additional Things to Keep in Mind

I didn't discuss this in detail here, but it's a good practice to have a "dead letter queue" (DLQ) on each queue setup, so you can retry failed events. You can read more about how to do this in this <a href="https://www.danielleheberling.xyz/blog/dlq-messages/" target="_blank" rel="noopener noreferrer">blog post</a>.

In our example architecture, we focused on SNS -> SQS -> Lambda; however, there are many other ways that you can achieve a "fan out" architecture. For example: <a href="https://aws.amazon.com/eventbridge/" target="_blank" rel="noopener noreferrer">Amazon EventBridge</a> or <a href="https://aws.amazon.com/step-functions/" target="_blank" rel="noopener noreferrer">AWS Step Functions</a>.

Check out this <a href="https://github.com/deeheber/event-fork" target="_blank" rel="noopener noreferrer">GitHub repo</a> for a starter template using <a href="https://aws.amazon.com/serverless/sam/" target="_blank" rel="noopener noreferrer">AWS SAM</a> if you'd like to see how to implement this architecture.

Routing on the Edge

Danielle Heberling — Thu, 09 Sep 2021 22:12:03 GMT

At Koan, our application’s frontend is a React Single Page Application running in two distinct environments (Staging and Production).

In addition to viewing the Staging and Production versions of our frontend, we also need to serve up a version of the frontend based off of a git commit in our Staging environment. Doing this gives Koan developers a “live preview” URL to review what the frontend looks like after committing changes but before they’re merged.

Our Solution

Our solution has the following high level steps:

Code changes are merged into our main branch. This action kicks off our CI system
The CI builds the code and places the build artifacts (static HTML/JavaScript/CSS files) into S3 buckets
A CloudFront CDN is in front of one of those S3 buckets
Our staging app domain is pointed at this CloudFront CDN
On all origin requests to the staging app domain → a Lambda@Edge function serves a build-specific index.html with static references to the rest of the build

More Details on Build Artifacts

Our CI process delivers build artifacts into S3 at /commit/[commit sha].

When a developer wants to “live preview” their recent commit, they need to add /commit/<their commit SHA> to the end of our staging app domain.

Each index.html file in this S3 bucket references static assets (CSS/JS files) hosted on a separate frontend-builds subdomain. This domain pointed at a second CloudFront CDN with a second S3 bucket as its origin. Serving these as CDN-friendly, immutable assets saves significant compute (money) for resources that don't need Lambda@Edge.

Inside the Lambda "router" function

Whenever that developer requests a specific version of the app, the request hits CloudFront as an origin-request. Our Lambda@Edge function receives a message event from CloudFront and then proceeds to do the following:

Gets the git commit hash from the pathname in the request. If there isn’t a commit hash in the URL, then we assume we want the latest version.
Gets the requested index file
Returns the index file as the body for our response

Let's see some code

Gets the git commit hash from the pathname in the request

Whenever someone makes an HTTP request to the CDN, the CDN then sends an event object to our Lambda@Edge function. The shape looks something like this.

We then pull the pathname off of that event object:

const url = require("url")

exports.handler = (event, ctx, cb) => {
  const { request } = event.Records[0].cf
  const { uri } = request
  const urlString = url.parse(uri)
  const { pathname } = urlString

  // ...more code here
}

Now that we have our pathname (including the optional commit/<commit sha> fragment), we can extract our git commit hash by calling a getHash helper function.

If there isn’t a hash present in the pathname this means that we just want to serve up the latest version of the app, so we'll return null.

const getHash = (pathname) => {
  const components = pathname.split("/").filter(Boolean)
  if (components[0] !== "commit") {
    return null
  }
  return components[1]
}

...

Gets the requested index file

Now that we have our git commit hash (or the null default) from the pathname, let's pass that commit hash into another helper function to get the desired index file from our S3 bucket.

The variables that start with process.env are NodeJS's way of referencing environment variables on the Lambda function. We set these variables when the function was provisioned.

If the S3 object (index.html file) is missing, we handle that in the catch and log the error.

const AWS = require("aws-sdk")

const s3 = new AWS.S3({
  apiVersion: "2006-03-01",
  region: "us-west-2",
})

const getIndexFile = (hash) => {
  let key = `${process.env.ENV}.html`
  if (hash) {
    key = `commit/${hash}/index.html`
  }
  const params = {
    Bucket: process.env.BUCKET,
    Key: key,
  }
  return s3
    .getObject(params)
    .promise()
    .then((result) => {
      if (!result || !result.Body || !Buffer.isBuffer(result.Body)) {
        console.error("null data")
        return null
      }
      return result.Body.toString("utf8")
    })
    .catch((err) => {
      console.error(err)
      return null
    })
}

A possible next step to improve this might be using Lambda@Edge memory. Since the index file is immutable, we should only need to retrieve it from S3 once (or if Edge memory is dumped). https://aws.amazon.com/blogs/networking-and-content-delivery/leveraging-external-data-in-lambdaedge/

...

Returns the index file as the body for our response

All together the function's code will look something like this

const AWS = require("aws-sdk")
const url = require("url")

const s3 = new AWS.S3({
  apiVersion: "2006-03-01",
  region: "us-west-2",
})

const RESPONSE_HEADERS = {
  // Add desired headers here
}

const getIndexFile = (hash) => {
  let key = `${process.env.ENV}.html`
  if (hash) {
    key = `commit/${hash}/index.html`
  }
  const params = {
    Bucket: process.env.BUCKET,
    Key: key,
  }
  return s3
    .getObject(params)
    .promise()
    .then((result) => {
      if (!result || !result.Body || !Buffer.isBuffer(result.Body)) {
        console.error("null data")
        return null
      }
      return result.Body.toString("utf8")
    })
    .catch((err) => {
      console.error(err)
      return null
    })
}

const getHash = (pathname) => {
  const components = pathname.split("/").filter(Boolean)
  if (components[0] !== "commit") {
    return null
  }
  return components[1]
}

// The main handler function code
exports.handler = (event, ctx, cb) => {
  const { request } = event.Records[0].cf
  const { uri } = request
  const urlString = url.parse(uri)
  const { pathname } = urlString

  const hash = getHash(pathname)

  return getIndexFile(hash)
    .then((body) => {
      if (!body) {
        console.error(`could not find file: ${uri}`)
        cb(null, {
          status: "404",
          statusDescription: "not found",
        })
        return
      }
      cb(null, {
        status: "200",
        statusDescription: "OK",
        headers: RESPONSE_HEADERS,
        body,
      })
    })
    .catch((err) => {
      console.error(err, `error with request uri: ${uri}`)
      cb(err)
    })
}

Closing

While there are opportunities for improvement, this setup works well for our team, and we thought that sharing this approach might give you and your team some ideas to iterate on.

More recently, AWS released CloudFront Functions. Stay tuned as we evaluate if that’s a good solution for us to use instead of our existing Lambda@Edge functions. It’s highly possible we could re-architect this to completely bypass the S3 GET and/or further utilize the edge caching.

Thanks to Daniel Kaczmarczyk and RJ Zaworski for reviewing drafts of this article.

Note: This post was originally published on the Koan dev blog

The Secret to Getting More Done

Danielle Heberling — Wed, 14 Jul 2021 22:12:03 GMT

It was a cold and rainy day as I sat alone in my home office debugging Webpack config errors. No matter what I tried, the errors would not go away. My natural inclination was to "just get through most of them" before eating lunch. But as I fixed errors, more emerged. Do you know what eventually got me through these hang ups?

Taking a break.

As my hunger grew stronger, I decided to give in and went to lunch. Being able to step away and temporarily detach my mind from the task at hand was just what I needed to approach this problem with a fresh approach full of new things to try.

Time and time again, the act of taking a break has helped immensely both for getting meaningful work done and for my overall mental state.

Photo by <a href="https://unsplash.com/@sanderdalhuisen?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Sander Dalhuisen</a> on <a href="https://unsplash.com/s/photos/lunch?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

My top three activities while taking a break throughout the workday are:

Take a walk outside if weather isn't too bad.
Change my surroundings. This could mean working at nearby coffee shop or moving from a desk to a couch. The context switching required to get up and move also helps to refocus.
Read a book. Bonus points if it is a physical copy or on an e-reader. It's important to not stare at the same screen all day.

Some companies are planning to take this idea a step further and are piloting a four day work week. Personally, I'm really interested to see how the rise of remote/hybrid workplaces as a result of the COVID-19 pandemic affects workers' break frequency. Curious to see if it goes up or down. In the meantime, I'm happy I get to work on a team that facilitates working with purpose, built on a culture that supports transparency, autonomy and inclusivity.

Four-day weeks and hybrid workplaces don't mean less work—just a more honest accounting of what already goes on. The standard work week in the USA is currently 40 hours, but no one is actually productive that entire time. I set up a poll on Twitter to get some data points on this and here's the results.

Remembering to take breaks has helped me to get more done while working fewer hours, and I can support my teammates better by bringing my best self to my work. Maybe it can help you too. What are some of your favorite activities to do when taking a break during the work day?

Note: This post was originally published on the Koan dev blog

Build a GraphQL API with TypeScript, AWS AppSync, and CDK

Danielle Heberling — Sun, 13 Jun 2021 22:12:03 GMT

What We're Building

Previously, I had used <a href="https://aws.amazon.com/serverless/sam/" target="_blank" rel="noopener noreferrer">AWS SAM</a> to build a <a href="https://github.com/deeheber/note-service" target="_blank" rel="noopener noreferrer">CRUD app for notes</a>. Recently, I wanted to learn more about the <a href="https://aws.amazon.com/cdk/" target="_blank" rel="noopener noreferrer">AWS Cloud Development Kit (CDK)</a>, <a href="https://graphql.org/" target="_blank" rel="noopener noreferrer">GraphQL</a>, and <a href="https://aws.amazon.com/appsync/" target="_blank" rel="noopener noreferrer">AWS AppSync</a>, so I decided to refactor my original project.

Disclaimer that at the time of writing the <a href="https://aws.amazon.com/blogs/developer/announcing-aws-cloud-development-kit-v2-developer-preview/" target="_blank" rel="noopener noreferrer">CDK v2 is in developer preview</a>. This blog post will use CDK v1.

If you want to skip ahead, take a look the finished project <a href="https://github.com/deeheber/note-service-next-generation/tree/blog-post" target="_blank" rel="noopener noreferrer">here</a>.

Setup

If this is your first time using the CDK, bootstrap your AWS account. The Prerequisites section in <a href="https://docs.aws.amazon.com/cdk/latest/guide/getting_started.html" target="_blank" rel="noopener noreferrer">this guide</a> has the steps.

Start by installing the CDK npm package and running the <a href="https://docs.aws.amazon.com/cdk/latest/guide/cli.html" target="_blank" rel="noopener noreferrer">cdk init</a> command. This is helpful because it gives you a directory structure along with a very minimal stack to get started.

On top of the initial scaffolding in /lib/note-service-stack.ts let's add some extra imports.

import {
  CfnOutput,
  Construct,
  RemovalPolicy,
  Stack,
  StackProps,
} from "@aws-cdk/core"
import { FieldLogLevel, GraphqlApi, Schema } from "@aws-cdk/aws-appsync"
import { AttributeType, BillingMode, Table } from "@aws-cdk/aws-dynamodb"
import { Code, Function, Runtime } from "@aws-cdk/aws-lambda"

The notes will be persisted in DynamoDB, so let's also add the following CDK code to create a notes-table.

const notesTable = new Table(this, "NotesTable", {
  billingMode: BillingMode.PAY_PER_REQUEST,
  partitionKey: { name: "id", type: AttributeType.STRING },
  removalPolicy: RemovalPolicy.DESTROY,
  tableName: "notes-table",
})

Create the API

The next step is to set up our api like this.

const api = new GraphqlApi(this, "NotesApi", {
  name: "notes-api",
  logConfig: {
    fieldLogLevel: FieldLogLevel.ERROR,
  },
  schema: Schema.fromAsset("src/graphql/schema.graphql"),
})

The schema property points to a file in the repository. You can also provide this inline if your prefer. The schema needs to be well formed in order to have a successful deploy of the stack.

Between the API and DataStore

The basic flow for GraphQL is:

schema => resolver => data source

The schema is your API contract. It says here's my query and mutation definitions in this API and he's what I expect to receive back from this specific query or mutation. Each field in that schema definition can be connected to a resolver that connects to the datasource.

For example our Query to get a note by id looks like this

  getNote(id: ID!): Note

It takes an id parameter and expectes a Note to be returned. A Note is defined in the schema to look like this

type Note {
  id: ID!
  content: String!
  author: String!
  createdAt: String!
  updatedAt: String
}

Traditionally on AWS one would use <a href="https://docs.aws.amazon.com/appsync/latest/devguide/resolver-mapping-template-reference-programming-guide.html" target="_blank" rel="noopener noreferrer">Velocity Template Language (VTL)</a> to resolve GraphQL schema fields to a datasource; however, last year AWS released the concept of a <a href="https://aws.amazon.com/blogs/mobile/appsync-direct-lambda/" target="_blank" rel="noopener noreferrer">direct Lambda resolver</a>. There's definitely reasons to continue using VTL over direct Lambda resolvers that I won't get into here, but I personally was pretty excited about this development since it meant that I could directly resolve to a Lambda used as a data source without needing to write any VTL.

Create Resolver and Data Source

For demonstration purposes let's focus on the getNote Query. This is a query that a client will make to this API that will send a note id and receive the note back.

Keep in mind that you'll also need to do similar things for all other queries/mutations in your API (in this case it's listNotes query, createNote mutation, deleteNote mutation, updateNote mutation).

Let's start by adding some CDK code to create the Lambda function

const getLambda = new Function(this, "GetLambda", {
  functionName: "get-lambda",
  runtime: Runtime.NODEJS_14_X,
  handler: "get.handler",
  code: Code.fromAsset("src/get"),
  memorySize: 3008,
})

The code property points to the source code for this function in src/get. This will contain the code necessary to retrieve our note from DynamoDB.

We'll then want to add this Lambda as a data source to the api we previously created.

const getDs = api.addLambdaDataSource("getDatasource", getLambda)

Next let's create a resolver to resolve the getNote Query field in our schema to our Lambda data source.

getDs.createResolver({
  typeName: "Query",
  fieldName: "getNote",
})

The final step is to ensure our Lamba has proper permissions to perform operations on DynamoDB, so let's send it the table name as an environment variable and grant it read permissions.

getLambda.addEnvironment("TABLE_NAME", notesTable.tableName)
notesTable.grantReadData(getLambda)

Finished Project

I personally have really enjoyed using GraphQL and the AWS CDK in my projects and hope you do too. Check out the full example <a href="https://github.com/deeheber/note-service-next-generation/tree/blog-post" target="_blank" rel="noopener noreferrer">here</a>. Contributions are welcome.

You Are More Than Your Tech Choices

Danielle Heberling — Sat, 22 May 2021 21:12:03 GMT

If you know me, you've probably noticed that my technical specialities lately have been JavaScript and AWS Serverless.

As a software developer with a few years of experience, recruiters often message me trying to convince me to apply for a job opening. In one specific instance, I was actively job seeking and came across a very interesting recruiter message encouraging me to apply for a Python developer position.

...sure I've read Python before and know a tiny bit, but my imposter syndrome immediately kicked in and I wondered if I'd be good enough. I then started reflecting on who I am as a developer and if I'd miss being primarily a JavaScript developer.

Looking Around

In my experience both online and offline when you get a group of software developers together, people have opinions. Strong opinions that sometimes transform into a way that people self identify as a developer.

Here's some examples of said opinions:

Monolith vs Microservices
Containers vs Serverless
Language X vs Language Y
SQL vs NoSQL
Don't use language X because it's old and bad

...ok I over emphasized a bit on the last one, but you get my point.

Some of these online and offline personalities even get extreme to the point of identifying as a "JavaScript developer", "the GraphQL guy", etc etc. and then only speaking on that topic.

Self Reflection

My initial hesitation of applying for that Python job got me thinking "how much of my identity as a developer is tied to the technological choices and tools that I use?"

...turns out it was quite a lot.

There's so many choices for tooling out there that it's easy to forget the "who", "what", and "why" by jumping immediately to the "how" when starting on a coding project.

Non-software engineer customers don't care if you used the latest and greatest framework or programming language to build that app, they care about the app working when they need it.

So this got me thinking more about what really matters to me as a developer.

What Really Matters (to me)

We're more than the tools that we choose to use as software developers. Technology is always changing, so those latest and greatest tools you're using today might not be the greatest in a few years. Instead of focusing on the tools, we should be focusing on why we're here.

One thing that will always be around are 1) coworkers and 2) customers, so I've shifted my professional focus to center around these two things.

When selecting tools to build something, I've changed my thinking towards trying to use the right tool for the right job. To me "the right tool for the right job" boils down to alignment with:

team values
team goals
the current team's strengths/weaknesses
being able to properly balance building out new features quickly and being able to maintain code that is already there

We all will naturally have specialities and interests and that's ok! But instead of identifying as a "JavaScript developer" I think from now on I choose to identify myself as "someone who enjoys team collaboration and solving real world problems for customers."

Upgrading to the AWS SDK for JavaScript v3

Danielle Heberling — Thu, 04 Feb 2021 22:12:03 GMT

In December 2020, AWS announced <a href="https://aws.amazon.com/blogs/developer/modular-aws-sdk-for-javascript-is-now-generally-available/" target="_blank" rel="noopener noreferrer">general availability</a> of version 3 of their SDK for JavaScript. I decided to take some time to experiment and see what's changed.

What's New?

There are lots of new features in this release. Here's the new features that I've seen highlighted the most.

Modular Packages

In prior versions, we'd just npm install the entire aws-sdk package and everything was in there. Now the sdk is split among multiple npm packages. Just install the packages that you need to use for your application. This is a big win for being able to get those code bundle sizes down.

Middleware Stack

The Middleware Stack gives developers more control over the lifecycle of the requests sent via the aws-sdk. My internal mental reaction is similar to how I felt after hearing the <a href="https://aws.amazon.com/blogs/compute/introducing-aws-lambda-extensions-in-preview/" target="_blank" rel="noopener noreferrer">AWS Lambda Extensions announcement</a>. It seems interesting, but I'm not quite sure what I'd do with it. That doesn't mean this isn't an exciting feature to someone else.

First Class TypeScript Support

TypeScript is growing in popularity, so it seems fitting that AWS would continue the trend. As someone who recently started using TypeScript, this is pretty exciting.

Let's See Some Code!

Because I was curious about how everything might look in v3, I experimented by converting a minimally featured CRUD api to use the new version. Check out the <a href="https://github.com/deeheber/note-service/blob/master/README.md" target="_blank" rel="noopener noreferrer">README</a> to see the high level architecutre of the app. And <a href="https://github.com/deeheber/note-service/pull/4/files" target="_blank" rel="noopener noreferrer">here's the git diff</a> when converting from the sdk v2 to v3.

Since DynamoDB is my database in this app, I went looking for the v3 equivalent of the <a href="https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB/DocumentClient.html" target="_blank" rel="noopener noreferrer">DynamoDB document client</a> that strips out the DynamoDB types and makes things more human readible. I found some mixed messaging on the official AWS Github repo for the JavaScript SDK and my overall takeway is that originally an AWS employee said they'd port it over, now it looks like that probably won't happen. My personal opinion is that they could've done a better job communicating <a href="https://github.com/aws/aws-sdk-js-v3/issues/1223" target="_blank" rel="noopener noreferrer">in this issue</a> filed on Github.

Anyway long story short, I found I was able to get the same result by marshalling and unmarshalling JavaScript objects/DynamoDB Records in the code. Info on that can be found in the docs for the util-dynamodb package <a href="https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/modules/_aws_sdk_util_dynamodb.html" target="_blank" rel="noopener noreferrer">here</a>. Another example can also be found in my git diff mentioned above.

Post update March 7, 2021 Looks like there's now a <a href="https://github.com/aws/aws-sdk-js-v3/pull/2097" target="_blank" rel="noopener noreferrer">DynamoDB Document Client</a> 😀.

What's Next?

Out of caution, I plan to wait for things to settle a bit more before doing upgrades on major business critical applications.

This has been a fun experiment to see what's changed, what stayed the same, and what I can look forward to in the future. Overall, I felt like AWS did a great job with this rewrite.

What are your thoughts on the update? Have any fun use case ideas with the new middleware stack feature? Let me know!

5 Things I Learned from Advent of Code

Danielle Heberling — Wed, 30 Dec 2020 22:12:03 GMT

Every year since 2015, <a href="https://adventofcode.com/" target="_blank" rel="noopener noreferrer">Advent of Code</a> happens. Each day from December 1st through December 25th, there's a new 2 part puzzle to solve.

On a typical year, I get to about day 4 then life gets in the way. When I finally remember there's more puzzles to do, it's almost time for the next year of puzzles. This year was pretty different in many ways, and I found myself with lots of free time. I acknowledge that not everyone might find this type of thing fun and that's ok, but this type of activity was perfect for me in my quarantine boredom. Here's my top 5 takeways from doing the 2020 edition of Advent of Code.

1. Think through how you would solve the puzzle in real life before writing code.

I found that sitting down with a piece of paper manually going through the puzzle sample input and seeing what had to be done to match the desired soltuion to be incredibly helpful. It's very tempting to jump straight to code, but I discovered when taking that route I'd often misunderstand a part of the instructions and go down the wrong path. Really thinking through the solution by writing down the steps in plain English and then in code proved to be the most successful method for me.

2. Puzzle input can be different between the sample and the full input.

Each day the puzzle provides you with sample input (sometimes a few sample inputs) with the answer to help you to get to the solution. There were a few days that left me scratching my head as to why the smaller sample input worked while the full input didn't. For the most part the sample input was pretty close to the full input, but these tiny differences could make or break my final solution. If you have outstanding questions in regards to the requirements, take a quick peek at the full input to see how it compares to the sample input and be sure you're taking everything into account. One day I spent hours assuming the input was a square when in reality it was a rectangle.

3. Ask for help.

The overall theme seems to be that the puzzles get harder as the days progress. The great part about these puzzles in my opinion is that they're focused on solving a problem in a programming language agnostic way. There's also a large community of folks who are putting their solutions up on Github, doing live streams on Twitch, or posting videos on YouTube. If you get stuck, there's no shame in taking a look at those to get ideas on how to continue or talking to someone else you know who's also doing Advent of Code regardless of what programming language they're using.

4. There's often multiple ways to solve the same problem.

I often found myself solving a puzzle and then looking at Github to see how others solved it. I learned a lot doing this too, since sometimes others solved the puzzle in a way that I would have never imagined.

5. Sometimes you need to know a random math theorem to solve the problem.

Day 13 part 2 was giving me some serious trouble. After hours of trying to get it to run, I finally gave up and took a look at someone else's solution. Turned out I needed to know some random math theorem in order to get this working. Not all of us are mathmeticians, so if you run into this don't get down on yourself. It's a great opportunity to learn something new.

Closing

In the year 2020, I set out to learn more about traditional computer science data structures and algorithims. Advent of code was a great way to take what I learned throughout the year while having some fun solving silly puzzles on the internet. It can be a tad stressful since there's a new puzzle each day, but don't let that get you down. It's ok to take a break and go back to those puzzles later. As of the time writing this, I have nearly completed all challenges and have already learned a lot. Check out my progress <a href="https://github.com/deeheber/advent-of-code/tree/master/2020" target="_blank" rel="noopener noreferrer">here on Github</a>. If you enjoy programming puzzles and want to learn something new while having fun, I encourage you to check out Advent of Code.

Using GitHub Actions to Build a Self Updating README

Danielle Heberling — Sun, 25 Oct 2020 22:12:03 GMT

The Project Idea

A few months ago, Github enabled the ability to add a README section to your profile in addition to the pinned repositories.

Personally, I find this pretty exciting. I'm sure many other software engineers can relate that we have some throw away projects and things that we built years ago on our profiles that don't fully reflect the big picture of our abilities or what we're currently into building. This gives us an extra place where we can highlight whatever we find important that we want to convey to people looking at our Github profile.

I decided to embark on the the journey of getting mine set up and wanted the following:

Work information (past and present)
Interests within the Software Engineering realm i.e. what languages, frameworks, etc I enjoy
What I'm currently excited about learning
Links to my website and twitter
Showcase my three most recent blog blog

Getting Started Building

Github makes this pretty easy to get set up...with that said, you can make this as hard or as easy as you choose. It is definately easy to add complication to make things challenging.

The base idea is that you create a repository named the same as your username, add a README.md, and anything you put in that README.md file will be showcased on the main page of your profile for others to see. I got that part done pretty fast and added some emojis to make it fun.

Going Beyond Markdown

I decided I didn't want to manually update my README file everytime I published a new post. I decided to automate it in order to ensure my three most recently published blog were showcased. Becuase I wanted a relatively lightweight solution that would be quick to implement, I decided to try out Github actions.

So the idea was to setup a job that runs once a day that does the following:

Clones my repository code
Sets up Node (my preferred language for this task)
Runs a script that
- Pulls down my three most recent post titles and links from my blog's rss feed
- Replaces the links in the current README file with the new information
If there are changes, push up the changes

Setting up the Job

In my repo, I clicked actions and set up a new workflow. This consisted of adding a .yml file to .github/workflow in my deeheber repo. Github had tons of examples for me to take a look through and the syntax was quicker to pick up than say something like AWS CloudFormation.

In the end that file was pretty readible and looked like this

name: Build README

on:
  workflow_dispatch:
  schedule:
    - cron: "30 15 * * *"

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Check out repo
        uses: actions/checkout@v2
      - name: Use Node
        uses: actions/setup-node@v1
        with:
          node-version: "12.x"
      - name: Install node dependencies
        run: npm install
      - name: Check for RSS feed updates
        run: npm run scrape
      - name: Commit and push if changed
        run: |-
          git diff
          git config --global user.email "actions@users.noreply.github.com"
          git config --global user.name "README-bot"
          git add -A
          git commit -m "Updated content" || exit 0
          git push

Even if you aren't familiar with the syntax, I think you can for the most part tell what each step is doing. This gets positive marks in my book.

The workflow_dispatch event allowed me to go into the Github UI under actions and hit a button to manually run this job to test that things were working...this was super helpful in my development workflow.

Also since this action was setup under the README Github repo, I didn't need to worry about providing Github credentials to the job. It just worked ™️.

Reflections

I am really impressed with how easy this was to get started and how quickly I was able to get something together and working. This also got me thinking that even though this is a super simple use case, that Github actions might be helpful for more complex cases and might reach for them in the future.

Here's how mine turned out

Here's my code if you want to take a look and try to build something similar yourself.

I'd encourage you to check this out. Even if you aren't in the DevOps yaml writing land like I am...I think Github has done a fantastic job with documentation to make this approachable for newcomers.

Dangers of Console-Driven Development

Danielle Heberling — Thu, 15 Oct 2020 22:12:03 GMT

What is Console Driven Development (CDD)?

AWS offers the ability to login to a web UI dashboard. In this dashboard, you can add, edit, and deploy various cloud resources. When I was first getting started with AWS, this is where I began for two reasons:

It seemed the easiest way to get started
Many tutorials out there included setting up cloud resources via the console

Story Time - AKA Real Life CDD

My very first full-time job as a Software Engineer was on a small enough team that all of our infrastructure was setup using the AWS console in a single AWS account. When I arrived everything was already established for the most part, so I just needed to create an EC2 instance to use as my dev sandbox for my dev version of the application. Easy enough until it was not easy.

I quickly learned that parts of the application went beyond my dev sandbox and used various AWS services that I had never heard of before nor knew how to set up. When a customer wrote in with a question or problem, it was often in these parts of the application. Not only that, I was also unclear how to test these parts of the application unless it was directly in our production environment.

Since the team had a lot of siloed knowledge in this area and we wanted the ability for everyone to have visibility into this part of the application, we hired a Cloud Engineer who began the task of transforming all of our existing infrastructure into deployable AWS CloudFormation templates. Even though at the time I didn’t have full grasp on what the templates meant, I was able to take a look at them, compare them with the AWS documentation, and make a reasonable conclusion about what that part of the application was doing.

Infrastructure-as-Code For Everyone

While getting started in AWS or any cloud provider, it can be easy to just jump straight to clicking through the console to create your resources. Short term that could work; however, as the application grows and/or you add new team members it can be difficult to remember exactly how you clicked through the console and configured resources. This is where the idea of infrastructure-as-code (IaC) comes into help.

IaC is helpful for many reasons, including but not limited to:

Have a written record of your infrastructure settings and how to connect things. No need to recall how and where in the console you set something up. Bonus if you have this in git, since it makes it easier to rollback to previous versions should the unexpected happen.
Transportable infrastructure. Having the IaC template allows us to take what we have in one cloud provider account and replicate it in another cloud provider account. Having separate dev, staging, prod etc. accounts helps reduce the risk of taking something down in prod when you just want to test a code change in a sandbox environment.
Onboarding new engineers is easier. All they have to do to get their dev environment setup is to clone the repo and deploy it into their sandboxed cloud provider account.

How Stackery is helpful...

Auto generates the IaC template by dragging and dropping resources on our Design Canvas. If you do know CloudFormation or want to go further than our default values, you have direct access to edit the template
Visualize your AWS stack via our visualization tool
Deploys your code via the UI dashboard or our CLI
Cloudlocal debugging of Lambda functions

Getting started in serverless and IaC can feel very daunting at first. It can be tempting to just click through the AWS console to create your cloud resources like many of the “getting started” tutorials recommend; however, I encourage you to sign up for a free trial of Stackery and give us a try. Your teammates, customers, and future self will be happy you made that choice.

Note: This post was originally published on https://www.stackery.io/

Computer Science for Everyone - Queues

Danielle Heberling — Tue, 08 Sep 2020 22:12:03 GMT

Photo by Halacious on Unsplash

Motivations

TL;DR. I come from a non-traditional background and never knew basic computer science concepts before getting into the industry. As I interviewed for software engineering roles, I found this to be a blocker at some companies. So, I've decided to see what all the hype was about and do some self study to improve myself and want to share what I learned with the hopes that it may help others too.

This is part 2 of a ??? part series. If you're interested check out what we covered last time <a href="https://www.danielleheberling.xyz/blog/cs-bubble-sort/" target="_blank" rel="noopener noreferrer">here</a>.

Let's get to the second topic - Queues

What is a queue?

Queues are everywhere in our lives. In Noth America, we more commonly refer to them as "lines". You go to the coffee shop, a lot of time you wait in line. You go out to buy groceries, you wait in line to pay for your groceries. You're driving on the turnpike, you wait in line inside your car to pay the toll.

The idea of the line or queue also exists in the digital world. You click to print something on your computer, that print job enters a queue where it waits in line to be printed. The common theme on this type of data structure is the priority order in which jobs get handled. The first item to enter the queue is the first item to leave the queue. Much like in real life how we frown upon line jumpers that cut in line, queues also do not like this. In more official terms this is known as "FIFO" or "first in first out".

Queue operations

Enqueue - Add an item to the end queue
Dequeue - Take the item at the front out of the queue
Peek - Take a look at the first element in the queue without removing it

What does this look like in code?

There's many ways to go about simulating a queue. I'm going to use Javascript since tons of the CS resources I've found tend to lean on Java for code snippets.

// Create the queue
const queue = [];

// Add items to the queue (enqueue)
queue.push('a');
queue.push('b');
queue.push('c');

// Remove items from the queue (dequeue)
while (queue.length > 0) {
  const itemToProcess = queue.shift();

  ...process the item taken off of the queue here
}

Going further

On the surface queues seem pretty simple especially if you're comfortable with arrays. Oftentimes queues are used as a building block for something larger and have the potential to get complicated pretty quickly, depending on what you're doing.

Queues are often used for breadth first (also known as level order) tree traversal. If you don't know what that means, that's ok. I'll most likely cover that in a later post.

Lambda Layer Update

Danielle Heberling — Fri, 03 Jul 2020 22:12:03 GMT

Background

Earlier this year, I did a <a href="https://www.danielleheberling.xyz/blog/lambda-layer-example/" target="_blank" target="_blank" rel="noopener noreferrer">walk through demo</a> on how to use lambda layers with NodeJS. We build a helper function that takes in a string and returns the string backwards.

<a href="https://aws.amazon.com/blogs/compute/working-with-aws-lambda-and-lambda-layers-in-aws-sam/" target="_blank" target="_blank" rel="noopener noreferrer">Now that AWS SAM and the SAM CLI has support for building layers</a>, it's now easier to get this setup. The way in my prior post still works, but personally I recommend utilizing this method going forward.

Setup

In this demo, I am going to use <a href="https://www.stackery.io/" target="_blank" target="_blank" rel="noopener noreferrer">Stackery</a>. Be sure to have the following installed in order to guarantee this will work.

The lastest version of the SAM CLI
The lastest version of the Stackery CLI
Docker desktop (just needed for local invoke)

Add Lambda Layer Resource

We start by adding a layer resource to the canvas in Stackery and connecting it to the function(s) that we'd like to use the layer. For readability, I renamed my Layer Resource Logicial Id to PalindromeLayer.

Make sure that your layer resource is referenced properly and that any old layers you're no longer using are removed. You can find this in the Function settings or in your template.

Move Layer Source Code

Since the AWS SAM CLI can now build our layer for us, let's move the source code. Using my example, I moved my package.json and palindrome.js file into src/PalindromeLayer.

I then made sure my ContentUri correctly referenced this new location.

Deploy the Layer and Locally Invoke

Once everything is deployed, I can now iterate locally on my function even if it has a layer. Since the Docker container used to build/include the layer in my function execution now needs to do the building, we need to add --build to the command.

Here's the command that I ran to test my function locally, keep in mind it might be a tad different than yours.

stackery local invoke -e development -f Function --build

Additional

You can still deploy a Layer in another stack and reference it by ARN. Some organizations probably prefer this method, especially if that layer is being used by multiple functions in different stacks. To do this, set your Build Method to none in the layer settings and add the layer ARN in the function(s) settings that you'd like to use the layer.

Closing

Personally I like this method better because it eliminate the manual process of zipping and uploading my Lambda Layer and then manually updating the layer version in my SAM template.

Check out the full code example in <a href="https://github.com/deeheber/lambda-layer-example/tree/layer-resource" target="_blank" target="_blank" rel="noopener noreferrer">this repo</a>.

All of the official documentation from AWS about Lambda layers can be found <a href="https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html" target="_blank" target="_blank" rel="noopener noreferrer">here</a>.

Building My Own Jamstack

Danielle Heberling — Fri, 26 Jun 2020 22:12:03 GMT

The trend of the frontend and backend coming closer together is continuing and even has a name now...the <a href="https://jamstack.org/" target="_blank" rel="noopener noreferrer">Jam stack</a>.

Last year, I discussed my excitment about the backend and frontend slowly merging closer together and did a walk through on how to deploy a JavaScript based frontend to an S3 bucket with website hosting enabled.

Building and deploying websites still continues to be a very common use case with folks using AWS, but I personally wanted a more "push button" solution that would also allow me to connect my Stackery backend resources easily. So I've decided to iterate on what we built the last time to make it better.

What We Built Last Time

<a href="https://www.danielleheberling.xyz/blog/text-to-speech/" target="_blank" rel="noopener noreferrer">Previously</a>, I wrote an application that translates written text to speech. I was getting tired of having to recall the different API endpoints to hit and wanted a nice UI to better manage these files...so I got to work on building a frontend.

Then I got tired of manually building and uploading those files to S3 for website hosting, so I wrote a Lambda function to do this for me. Here's what the final architecture looked like

That is all fine and still works, but keeping up with latest Serverless trends I recognize that writing code in a Lambda function can cause potential unwanted technical debt. What if there was some service that I could just input specific things like:

source code location
build command
built files location
location (an S3 Bucket) to upload the built files

Lucky for us, there's AWS CodeBuild!

So I embarked on the journey of replacing my PopulateFrontend Lambda function with a CodeBuild Project.

The Easy Push Button Way of Doing this

Stackery released the <a href="https://docs.stackery.io/docs/api/nodes/Website/" target="_blank" rel="noopener noreferrer">Website Resource</a>. Check it out...it's just a matter of dragging and dropping the resource into your stack and providing a few settings (Source Directory, Build Command, and Publish Directory).

We have a more detailed tutorial <a href="https://docs.stackery.io/docs/tutorials/react-spa-tutorial/" target="_blank" rel="noopener noreferrer">here</a>.

Connecting my Frontend to a Backend API

Once your S3 Bucket and Website are on the canvas, add an API too (both HTTP API and original API Gateway work).

After you connect the API to the References facet under the Website, you'll then see that the API_URL environment variable is available to the CodeBuild Project.

I personally decided to write a minimal shell script that writes the API_URL environment varaible to a file that my frontend code later references. Here's a snippet.

#!/bin/sh

# Inject API into config file
echo "export default {
  backendAPI: '$API_URL'
};" > src/config.js

echo "Config file written to src/config.js"

# Build the site
npm run build

I added an npm script to my package.json that executes this script, titled it "production", and updated my Website Build Command to be npm run production. All this script does is run the snippet above.

Speeding up the Delivery with a CDN (Optional)

If you want to speed up the delevery of your website, I'd suggest adding a CDN in front of your S3 bucket using Amazon CloudFront. Connect your S3 Bucket to the CDN Origin like this.

The CodeBuild Job

The rest of this post is going to be a technical deep dive under the hood for those who are interested.

The CodeBuild job does the following:

Installs dependencies (in my case NodeJS and NPM)
Copies my source code into the build job container
Executes my build command
Copies the built files into an S3 Bucket

All of the necessary variables are set via the Stackery Website resource settings, but you can also directly alter the template if you prefer.

Here's what the CodeBuild job template code looks like for my text to speech converter app:

Website:
  Type: AWS::CodeBuild::Project
  Metadata:
    StackeryType: website
  DependsOn: WebsiteRole
  Properties:
    Name: !Sub ${AWS::StackName}-Website
    Artifacts:
      Type: NO_ARTIFACTS
    Environment:
      ComputeType: BUILD_GENERAL1_SMALL
      Image: aws/codebuild/amazonlinux2-x86_64-standard:3.0
      Type: LINUX_CONTAINER
      EnvironmentVariables:
        - Name: API_URL
          Value: !Sub https://${HttpApi}.execute-api.${AWS::Region}.amazonaws.com
    ServiceRole: !GetAtt WebsiteRole.Arn
    Source:
      Type: NO_SOURCE
      BuildSpec: !Sub
        - |-
          version: 0.2
          phases:
            install:
              runtime-versions:
                nodejs: latest
                python: latest
                ruby: latest
              commands:
                - |
                  _SOURCE_LOCATION=${SourceLocation}
                  if [ s3 != "${!_SOURCE_LOCATION%%:*}" ]; then
                    git clone ${SourceLocation} repo
                    cd repo
                    git checkout ${SourceVersion}
                  else
                    aws s3 cp ${SourceLocation} repo.tgz
                    tar --strip-components 1 -xvvzf repo.tgz
                  fi
                - cd ${SourceDirectory}
            pre_build:
              commands:
                - |
                  if [ ! -f yarn.lock -a -f package.json ]; then
                    npm install --production
                  elif [ -f yarn.lock -a -f package.json ]; then
                    yarn install --production
                  elif [ -f requirements.txt ]; then
                    pip install -r requirements.txt
                  elif [ -f Gemfile ]; then
                    bundle install
                  fi
            build:
              commands:
                - ${BuildCommand}
            post_build:
              commands:
                - |
                  _SOURCE_LOCATION=${SourceLocation}
                  if [ s3 != "${!_SOURCE_LOCATION%%:*}" ]; then
                    cd "${!CODEBUILD_SRC_DIR}/repo"
                  else
                    cd ${!CODEBUILD_SRC_DIR}
                  fi
                - aws s3 sync '${PublishDirectory}' 's3://${DestinationBucketName}' --acl public-read --cache-control 'max-age=0, must-revalidate, public' --no-progress --delete
        - PublishDirectory: src/frontend/build
          BuildCommand: npm run production
          SourceDirectory: src/frontend
          DestinationBucketName: !Ref FrontEnd
    Tags:
      - Key: Stackery Project Type
        Value: Website Builder

Orchestrating the CodeBuild Job

Now that the CodeBuild job is setup, we know how we're going to build the site and publish it to an S3 Bucket. The remaining question is how can we trigger the CodeBuild job to start when the CloudFormation stack is deployed?

My answer to that question was to use a <a href="https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources.html" target="_blank" rel="noopener noreferrer">Custom Resource</a>. The template for it looks like this

WebsiteBuildTrigger:
  Type: Custom::StackeryWebsiteBuildTrigger
  DependsOn: WebsiteEvents
  Properties:
    ServiceToken: !Sub arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:stackery-agent-commander
    Type: website
    ProjectName: !Ref Website
    SourceVersion: !Ref SourceVersion

Note the Type starts with Custom::...this is how CloudFormation knows it's a custom resource.

Any time one of the Properties changes (in this case the SourceVersion), CloudFormation will trigger Lambda function I specify under ServiceToken. In this case, it's a Stackery controlled Lambda function in a different CloudFormation stack that has two main jobs:

Trigger the start of the CodeBuild job
Report back to CloudFormation when the CodeBuild job succeeds or fails, so CloudFormation knows how to continue deploying the stack

Step 1 is handled directly in the Lambda function via <a href="https://docs.aws.amazon.com/cli/latest/reference/codebuild/start-build.html" target="_blank" rel="noopener noreferrer">this command</a>, but how can we enable the CodeBuild job to report back a success or failure to CloudFormation?

My answer is CloudWatch Events. This is wired up in the template under the resource named WebsiteEvents.

WebsiteEvents:
  Type: AWS::Events::Rule
  DependsOn: Website
  Properties:
    EventPattern:
      source:
        - aws.codebuild
      detail-type:
        - CodeBuild Build State Change
      detail:
        build-status:
          - SUCCEEDED
          - FAILED
          - FAULT
          - STOPPPED
          - TIMED_OUT
        project-name:
          - !Ref Website
    Targets:
      - Arn: !Sub arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:stackery-agent-commander
        Id: StackeryAgentCommander

When the CodeBuild Project for our Website has a build status of SUCCEEDED, FAILED, FAULT, STOPPED, or TIMED_OUT, the configured CloudWatch Event reports this information back to my custom resource function. My custom resource function then sends the appropriate information to CloudFormation. Once CloudFormation gets a pass/fail response, it knows how to conintue with deploying the stack.

Closing

Check out the entire code repository referenced <a href="https://github.com/deeheber/text-to-speech-converter/tree/blog-post-3" target="_blank" rel="noopener noreferrer">here</a>.

There are definitely many different approaches to build and deploy a JavaScript frontend on AWS. I encourage you to investigate everything that's out there and see what works best for you.

I'm biased, but I recommend checking out Stackery. Not only has the <a href="https://docs.stackery.io/docs/api/nodes/Website/" target="_blank" rel="noopener noreferrer">Website resource</a> been added recently, we've also updated our Developer plan to have more features still at the amazing price of free.

Computer Science for Everyone - Bubble Sort

Danielle Heberling — Fri, 10 Apr 2020 22:12:03 GMT

Photo by Pieter on Unsplash

Motivations

As an individual that is a professional software engineer and does not have a computer science degree, data structures and alogorithms used to be frightening words for me to hear. Oftentimes my inner voice translated those words into "here's some concepts that are often not used regularly, yet come up in software engineering interviews all the time. This gives us an excuse to not hire you solely based off of the fact that you do not have a computer science degree."

After being in the industry for a while, I've found my inner voice to be partially factual. The technical interviewing process is broken. As an individual who's had to interview others and had to write some coding challenges...I can totally understand why interviewers sometimes gravitate toward asking these types of questions beyond the aforementioned gatekeeping. It's nearly impossible to find a perfect set of coding questions to test one's knowledge...sometimes it's just easier to ask these types of questions to see how someone thinks through problems.

While I don't have a solution to how we can fix the broken coding interview process, I decided to focus on what I can control. I felt a sense of imposter syndrome, because I could not answer those types of questions in interviews...it was downright embarassing. Once an interviewer asked me to write a linked list and at the time I didn't even know what a linked list was. For those reasons, I decided to learn a bit about these concepts and hope to share over a series of blog what I've learned in hopes that it may help others.

Let's get onto the first topic

Bubble Sort and how it works

We have a list of items and want to put them in order. For example purposes and to keep things simple, let's just use a list of numbers.

8, 6, 7, 5, 3, 0 ,9

Here's the high level steps on how to sort this via the bubble sort method:

Start going through the list
Compare two items at a time
If the two items are out of order swap them
Once going through the entire list, if we swapped items --> repeat point #2 and #3
End with a list in order once no items are swapped

With our example list we'll start by comparing the first two numbers

8 and 6 are out of order --> swap their order
The list is now 6, 8, 7, 5, 3, 0, 9
Move to the next two items and compare 8 and 7
Out of order so swap their place
The list is now 6, 7, 8, 5, 3, 0, 9
Continue until you've gone through the entire list
If you swapped at least once, repeat the steps through the entire list

Code Snippet

Since a lot of computer science example material seems to be Java-centric, here's what a bubble sort looks like in JavaScript for a little change of scenery. Keep in mind there are other ways to write this.

function bubbleSort(nums) {
  let wasSwapped = false
  do {
    wasSwapped = false
    for (let i = 0; i <= nums.length - 1; i++) {
      const firstNum = nums[i]
      const secondNum = nums[i + 1]

      if (secondNum < firstNum) {
        nums[i] = secondNum
        nums[i + 1] = firstNum
        wasSwapped = true
      }
    }
  } while (wasSwapped)
  return nums
}

// Test case
console.log(bubbleSort([8, 6, 7, 5, 3, 0, 9]))

Time and Space Complexity

When talking about computer science-y things time and space complexity appears to be important, so I'd be remiss to not mention this.

In regards to "Big O" this has an average time complexity of On^2. Compared to other sorting methods it is rarely the most efficient, since one needs to go through the entire list at least once (often multiple times) to get a sorted list. It is not recommended to use in production, but since the though process generally aligns well with most people's natural thought process it is a good starter sorting algorithm used for teaching purposes.

It has a good space complexity of O(1) or "constant time".

Go forth and sort the bubbles

I hope to be back with some similar blog explaining other computer science concepts in the future.

Create Your Own Lambda Layer with NodeJS

Danielle Heberling — Wed, 22 Jan 2020 22:12:03 GMT

Photo by Ivana Cajina on Unsplash

The Problem

While speaking with folks who are new to serverless, I’m often asked “how can I reuse my code across multiple Lambda functions?” One way to do this is by writing your own custom Lambda Layer. In this post, we’re going to build a simple layer using nodeJS, deploy it, and use the layer in a function.

Note: there is an updated method for using Lambda layers outlined here

What We’re Building

We’re going to write a lambda layer that contains a helper function. This function takes in a word or phrase and reverses the string. It’s not overly helpful in prod, but a simple example can be useful in seeing how things are setup.

Write Lambda Layer Source Code

Create a new directory named palindrome. The file hierarchy will look like this.

palindrome
|_ nodejs
    |_ package.json
    |_ palindrome.js

The package.json will look like this

{
  "name": "palindrome",
  "version": "1.0.0"
}

The reverse.js file looks like this

module.exports = function (phrase) {
  return phrase.split("").reverse().join("")
}

Note that palindrome and nodejs are folders. The nodejs folder naming and placement is very important because it affects how the layer delivers the code to functions.

Deploy the Layer

First we’ll need to zip up the layer in order to upload it to the AWS Lambda console.

You’ll want to cd into your palindrome folder via the terminal and run:

Zip -r palindrome.zip ./*

You should now see a .zip file added in the nodejs directory. This is the file we will upload via the AWS Lambda console.

palindrome
|_ nodejs
    |_ package.json
    |_ palindrome.js
|_ palindrome.zip

You can then find the layer ARN here...copy/paste this for use later.

If you prefer to use the AWS CLI instead of the console, you can also do that by following <a href="https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html#configuration-layers-manage" target="_blank" target="_blank" rel="noopener noreferrer">these directions</a>.

<a href="https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-layerversion.html" target="_blank" target="_blank" rel="noopener noreferrer">Cloudformation</a> also offers the ability to set this up; however, be careful about creating the layer in the same stack that has dependencies on the layer. It’s my personal opinion that keeping dependencies in a separate repo/stack might be a better practice to imitate how npm and other package registries work.

Create a Function that Uses that Layer

I personally like to use Stackery in order to add my resources since it adds all the necessary permissions and connection for me via a drag and drop interface, so these instructions will show how to do that. Keep in mind if you wish to set it up using another method, be sure to remember to add the <a href="https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html#configuration-layers-permissions" target="_blank" target="_blank" rel="noopener noreferrer">correct permissions</a>.

Head over to the canvas and double click your function to open up this dialog. You’ll then want to paste your layer ARN here:

Note: you need to do this everytime you update the layer version.

Not overly necessary for this example, but if you have multiple environments that rely on different layers you can <a href="https://docs.stackery.io/docs/using-stackery/environments/#setting-configuration-store-values" target="_blank" target="_blank" rel="noopener noreferrer">setup a parameter</a> to reference this layer that will be dependent on which environment you’re deployed into.

Also keep in mind that as of my writing this, AWS has a limit of 5 layers per function.

Add code to the function

The code in your index.js should look like this:

const palindrome = require("/opt/nodejs/palindrome")

exports.handler = async (event, context) => {
  // Log the event argument for debugging and for use in local development.
  console.log(JSON.stringify(event, undefined, 2))

  const words = "This string should be reversed"
  const response = palindrome(words)

  return {
    statusCode: 200,
    headers: {},
    response: JSON.stringify(response),
  }
}

The important thing to note here is the first line where we require from /opt/nodejs, otherwise you can use your library like you normally would when developing outside of Lambda.

Once deployed, go ahead and invoke the function. You should see the original string reversed in the function response.

Check out the full code example in <a href="https://github.com/deeheber/lambda-layer-example/tree/original-blog-post" target="_blank" target="_blank" rel="noopener noreferrer">this repo</a>.

Serverless Can be for Everyone - Let's Build it that Way

Danielle Heberling — Mon, 23 Dec 2019 22:12:03 GMT

Serverless can bring opportunities by making DevOps more accessible to folks new to the industry. It can also bring opportunities to folks already in the industry that want to spend more time focusing on adding business value. But many technologists, seasoned or otherwise, hear a lot about serverless but don’t always know how to get started. As an early adopter of serverless, I am very interested in closing that gap by encouraging widespread adoption across all types of users and demographic groups. For me, this goal is a personal one. Serverless IS for everyone, but the onus is on those already involved in the movement to make opportunities and resources more available — that means me! I want to share a bit about my story and goals with the hopes that it might a) inspire you to see how serverless can do the same for you and b) open up a conversation about how people like me can help others interested in serverless.

How I Started

For those of you who don’t know me, a little introduction.

I began my adult career as a music teacher and then transitioned to tech support. Tech support was a wonderful experience because each day was a new challenge. Customers would write in with questions, bugs, and feature requests that would often require me to research their inquiry in order to fully address it.

After a few years of working in tech support, I noticed a pattern.

A customer would write in
I’d try my best to address their question/resolve their issue
Oftentimes, I’d be limited in how much I can help them out due to my role’s credentials and my experience-level
I’d escalate the issue to a software engineer that had the access and expertise to directly fix their issue and/or add their desired feature in the code.

Because I wanted to provide a higher level of support to customers, I decided I wanted to be that person who made the code change to directly address the customer’s inquiry vs just handing it off to another individual. I saw a need within my own day-to-day work and a clear way to address it.

For this reason, I quit my day job and attended a code school to become a software engineer.

My Engineering Journey Begins

In code school, we spent a lot of time learning JavaScript. One thing that was lacking and became apparent whenever it came time to do our final group project was how to deploy the applications we were building. To be fair, the program was very fast-paced and it was impossible to cover everything, so in the meantime, my team used <a href="https://www.heroku.com/" target="_blank" rel="noopener noreferrer">Heroku</a> to deploy our final project. That worked well, but as an individual, I often found myself pondering how this would look with a production-level enterprise application. What’s the point of developing an application if you don’t know how to share it with the rest of the world?

Fast forward to my first job post-code school graduation. The team used Docker containers very heavily to deliver our application to the end-users. Since there were so many microservices and moving parts, we also used <a href="https://aws.amazon.com/ecs/" target="_blank" rel="noopener noreferrer">ECS</a> to handle the management of the container clusters.

On an individual level, I loved my job and really enjoyed writing code to solve problems. One thing that was necessary for my job was writing Dockerfiles and managing the various ECS clusters that were deployed on AWS. I wasn’t actually very interested in this part of the position but, since we were a small team, we had a weekly rotation where a different engineer every week was assigned deployment duty. Sometimes during deployment duty, I’d poke around AWS and see various managed services like SQS, SNS, Lambda etc. I had no idea then that my interest in wanting to know what these did would affect me down the line.

One day, my boss asked me to review a pull request in which he was experimenting with using a serverless architecture to create a chatbot. The idea was to experiment with a smaller tool used internally to see if this type of architecture might work for customer-facing production applications. I immediately had many questions and became hooked on this type of application building.

Changes in Development Workflow and Minset Post-Serverless

Fast forward a few months. At this point, I was no longer focused on provisioning and managing Docker containers. Instead, I was utilizing managed services (in this case AWS) to do this work for me. Now that I had the extra mental bandwidth, I could spend time focusing on writing business logic. A mentor once told me that as a software engineer we’re paid to solve problems not to write code — and since I had begun to digest the concept of serverless, I had a better grasp on what that exactly meant.

Here’s a few highlights of things that noticed from a technical standpoint that needed to shift:

Viewing logs in the cloud provider can be very helpful in troubleshooting code not working as expected.
Local testing is possible with tooling, but it doesn’t completely replace deploying your application and testing it out in the cloud.
Service limits are now tied directly to the managed services being used. Usually, these are very high limits. If you max out on a service limit, most cloud providers can raise the limit for you if you ask.
Event-driven models and async workflows are more common.
Deployment strategy / Environments / Infrastructure as code is more important than ever.

What's Next for Serverless

Many people mark the start of serverless with the launch of AWS Lambda in late 2014. If you think about it, it really wasn’t that long ago. Serverless as a concept is still in its early stages, so is the community surrounding it.

I do a lot of time thinking about the serverless community, the degree to which it’s evolved in this short amount of time, and the direction I believe it should go in. It might sound like an oversimplification, but I believe the most important thing that we need to intentionally focus on in regards to evolving the serverless ecosystem is inclusion. We must do everything possible to attract folks from all backgrounds in order to make the community stronger and to build better applications that more accurately reflect the diverse user base we serve. It’s also crucial to continue asking what that looks like: how do we speak to various individuals and groups? It’s not a static answer.

For now, I’ve come up with an answer in my own personal work and life: speaking to as many different groups in as many different locations as possible. Earlier in December, I spoke alongside Stackery’s Ecosystem’s Director, Farrah Campbell at <a href="https://www.stackery.io/blog/reinvent-danielle-talk/" target="_blank" rel="noopener noreferrer">AWS re:Invent in Las Vegas</a>.

This was an amazing and humbling opportunity that I’m still stoked about! But it’s no secret that not everybody is able to attend a huge and costly event like re:Invent for both logistical and financial reasons. So what other communities can I speak to and with?

Serverless Days Nashville 🤠

Next up is Tennessee for <a href="https://serverlessnashville.io/" target="_blank" rel="noopener noreferrer">Serverless Days Nashville</a>. The topic? Encouraging people to take the first (or umpteenth) step in their serverless journey by using Stackery. If you can/will be in the area in February, I’d love to see you there. Want to connect about serverless and Stackery (or share any sweet tips about Nashville)? <a href="https://twitter.com/deeheber" target="_blank" rel="noopener noreferrer">Reach out to me on Twitter</a>.

Note: This post was originally published on https://www.stackery.io/

Real Time Interactions with Websockets

Danielle Heberling — Tue, 19 Nov 2019 22:12:03 GMT

Photo by Quino Al on Unsplash

The Problem

I was working on a web application that allows users to upload plain text files and translate that text into another language (with the caveat that the inbound and outbound languages are supported by <a href="https://docs.aws.amazon.com/translate/latest/dg/what-is.html" target="_blank" rel="noopener noreferrer">AWS Translate</a>). The frontend talks to the backend via AWS API Gateway REST endpoints.

Everything was going well, until I started uploading longer text files to translate. Because of the larger file size, the backend took a while longer to process and I found my HTTP connection response was timing out before the translation was complete. Sure refreshing the webpage until the translation was complete is a solution that would work...but I wanted something a little more seamless.

Enter websockets. Websockets allow for two way communication between clients (in this case my website in the browser) and a backend (in this case my AWS API, Lambda Functions and S3 Buckets). A popular use case for websockets is real time chat applications. Someone writes and sends a message and the message is received by everyone in that particular chat room.

In my particular use case, I wanted to send a request to start the translation from my client (the website) to my backend. Whenever the translation is ready, send a response to the client from the backend (it is ready when the translated file is put in an S3 Bucket).

Setting Up the Websocket API

Luckily Stackery offers the Websocket API Gateway <a href="https://docs.stackery.io/docs/api/nodes/WebSocketApi/" target="_blank" rel="noopener noreferrer">as a drag/drop resource</a> in the editor. So I started by doing this. The important thing for me to add was my routes. AWS has three reserved routes $connect, $disconnect, and $default. It’s up to you if you want to include them in your project or not. $connect is triggered when a client connects to the websocket API, $disconnect is triggered when a client disconnects, and $default is the route that requests fall back on if a request comes in that doesn’t match any of the routes. You can also add your own custom routes in addition to these three default ones.

The default route key selection mechanism is $request.body.action, so I decided to stick with that. This means that in the body of your requests, you need to include an “action” key that has a value of one of your routes in order to ensure your request gets sent to the desired route in your API. You can also include any other important key/value pairs that your system is expecting. For example, I want to send some data to a route named "test", the body of my request would look something like this:

{ “action”: test, “data”: “data I am sending” }

Backend walkthrough

On the other end of each API Gateway route, I connected a Lambda Function. I set up my $connect route to store all client connection ids in a DynamoDB table and my $disconnect route to remove the connection id from the DynamoDB table when a client disconnects. This is a nice way of keeping track of who is all connected, especially if I want to broadcast changes to all or some of my clients.

Next I got to work on my BucketPutNotification Lambda Function. I first wired it up to be triggered on s3:ObjectCreated:* and s3:ObjectRemoved:* events.

This Lambda Function works like this:

Get information about item(s) placed or removed from the S3 Bucket from the event object passed into the function.
Get all connection ids from the DynamoDB table.
Send the S3 Bucket information to every client.

In order to send my notification from my Lambda Function to my client(s), I used the postToConnection method off of the <a href="https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/ApiGatewayManagementApi.html#postToConnection-property" target="_blank" rel="noopener noreferrer">ApiGatewayManagementApi JavaScript class</a>.

Keep in mind that your Lambda Function must have execute-api:ManageConnections IAM permission in order to be able to do these types of sdk calls. It will also need to know your API connections endpoint as well as your client connection ids. One hiccup I had was neglecting to stringify the data I was sending. According to the documentation, the data being sent must be a string or buffer. I can tell you firsthand if you’re trying to send a JavaScript object, it won’t work.

<a href="https://www.npmjs.com/package/wscat" target="_blank" rel="noopener noreferrer">Wscat</a> was a tool suggested throughout the AWS docs in order to setup a client to test your websockets. I used this for testing purposes, but my end goal was not a CLI type of interface.

Here's what my backend ended up looking like from an architectural standpoint.

(click to expand image)

Client (Frontend) Walkthrough

For this particular project my desired client was a website built using reactJS, so here’s how I went about getting that setup.

I first grabbed my Websocket URL from the Stackery dashboard (can also get it from the AWS console) and used the browser <a href="https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API" target="_blank" rel="noopener noreferrer">Websocket API to connect</a> via new WebSocket on ComponentDidMount.

Once connected, my application listened for specific events onopen, onmessage, onclose, onerror in order to determine what type of action to take. These events (as well as others) are outlined in the <a href="https://developer.mozilla.org/en-US/docs/Web/API/WebSocket" target="_blank" rel="noopener noreferrer">Websocket API spec</a>.

If I want my frontend client to send a message to the backend, I can use the send method off of my connected WebSocket object. Then I’d include a JSON object that looks something like this:

{ “action”: “echomessage”, “data”: “data I am sending” }

Closing

We’ve only scratched the surface, but hopefully this post can give you some insight on how to setup websockets to talk to a web ui client via API Gateway websockets. If you’d like to see a working example of the infrastructure/code, I discussed in this post take a look at <a href="https://github.com/deeheber/websockets" target="_blank" rel="noopener noreferrer">this GitHub repo</a>.

AWS Amplify Review

Danielle Heberling — Wed, 06 Nov 2019 22:12:03 GMT

Photo by Kevin Ku on Unsplash

Recently, I’ve been playing around with <a href="https://aws.amazon.com/amplify/" target="_blank" rel="noopener noreferrer">AWS Amplify</a>. Wanted to share some thoughts in case anyone is interested in exploring this ecosystem. Disclaimer that these are my opinions and experiences. You may have a different one. This is ok.

Why Did I Do this?

If you talked to me three years ago, I would’ve labeled myself as a software engineer who specializes in frontend. I embraced this label until I was introduced to the world of serverless. All of a sudden, I realized that I could write infrastructure as code to use managed services to create more powerful systems. Systems that would auto scale to accommodate various workloads while not having to reinvent the wheel. No longer did I need to be incredibly bored writing Dockerfiles and orchestrating clusters of containers.

Disclaimer that serverless is much much more than auto scale and utilizing managed services, but this blog post isn’t on that topic. I highly recommend <a href="https://read.acloud.guru/serverless-is-a-state-of-mind-717ef2088b42" target="_blank" rel="noopener noreferrer">this blog post</a> if you want to get into that topic. Also disclaimer about Docker/containers etc. I have a strong belief that you should use the right tool for the job and sometimes that tool happens to be containers. Just from my personal experience, I haven’t gotten that much joy out of writing Dockerfiles etc.

This world of serverless also brought along a heavy focus on using and learning about various AWS services. It’s been a fun ride, but I do still find myself enjoying jumping into the frontend of an application every now and then, so the aws-amplify NPM package was a great utility library to use when connecting my frontend to various AWS services. With all of the new features of AWS Amplify Console, I decided to take a deeper look to see what this ecosystem is like.

What I Built

Since I learn best by doing, I decided to build the classic TODO application with AWS Amplify. This would utilize CRUD (create, read, update, delete) endpoints talking to my <a href="https://reactjs.org/" target="_blank" rel="noopener noreferrer">ReactJS</a> frontend. As a bonus, I decided to add authentication. Authentication and Cognito are two things that have always been pretty difficult to implement. This seems to be true with some of my conversations with pretty senior Software Engineers too.

GraphQL API

When first setting up the API, I noticed an option for <a href="https://graphql.org/" target="_blank" rel="noopener noreferrer">GraphQL</a>...I figured why not let’s go down this journey even further. After a few prompts from the Amplify CLI and an amplify push...I had a CRUD GraphQL API setup complete with schema generation, mutations, queries, subscriptions, resolvers, and a DynamoDB table as my data source. If you would’ve asked me about some of those words a few months ago, I wouldn’t have been able to tell you what they mean...but AWS Amplify made it super easy to setup, deploy, and learn more about those things. I also enjoyed playing around in the Appsync console creating test mutations and queries to experiment more with GraphQL.

Setting up Authentication with Cognito User Pools

After getting a CRUD api setup for my TODO app, it was time to add authentication. This way someone would need to login to see the TODO items, and they could only create, read, update, and delete their own TODO items.

I first ran amplify add auth in the console and followed the prompts to get setup. There are options where you can authenticate with social media logins, but I decided to use Cognito User Pools. You can also customize the sign in workflow.

After another amplify push, I now needed to update my GraphQL schema to only allow the owner of the TODO item to CRUD it. Luckily AWS Amplify has an @auth directive that does just this.

Here’s what my schema looked like before:

type Todo @model {
  id: ID!
  name: String!
  complete: Boolean
}

And after

type Todo @model @auth(rules: [{allow: owner}]) {
  id: ID!
  name: String!
  complete: Boolean
}

You can also make this more complicated if you want. There are also the options to only allow specific groups, only allow certain types of operations (i.e. read, update etc), and add field level auth rules too. Probably more that I didn't end up using.

I also added a login screen by using the aws-amplify withAuthenticator higher order component. I chose not to customize it, but it is flexible to allow for customization (https://aws-amplify.github.io/docs/js/authentication#customize-withauthenticator). Overall it took me minutes to setup what once took me hours (and sometimes days) to do.

Frontend Deploy

To host the frontend on AWS, the choice most folks go with is uploading their frontend code to an S3 Bucket and sometimes adding a CloudFront CDN in front of the S3 Bucket. This seems like it should be simple to automate, but it really isn't quite yet. With Amplify it’s easy. All you need to do is amplify add hosting and amplify publish then you’re done! Much nicer than writing a <a href="https://www.danielleheberling.xyz/blog/deploy-frontend/" target="_blank" rel="noopener noreferrer">weird Lambda Function</a> to do this.

Summary

Amplify is a great choice if you want to get started with GraphQL/Appsync. It’s also good for frontend focused applications that needs to utilize a backend built using AWS. Overall, I’d recommend it to folks who are new to AWS or seasoned folks who want to get a project up and running quickly. There are also great CI/CD tools you can use with the <a href="https://aws.amazon.com/amplify/console/" target="_blank" rel="noopener noreferrer">Amplify Console</a> which I didn’t touch on in this post. Major bonus that the examples in the AWS Amplify docs are using React hooks.

A downside that I noticed was there didn’t seem to be a way to eject from the framework. This could be problematic if down the line I decided I wanted to use <a href="https://aws.amazon.com/serverless/sam/" target="_blank" rel="noopener noreferrer">AWS SAM</a> or the <a href="https://serverless.com/" target="_blank" rel="noopener noreferrer">Serverless framework</a> instead. Also as someone who likes to understand how things work in the event of error messages...it seemed like AWS Amplify performed a lot of “magic” under the hood and I could not discern how the framework was doing things. It would be amazing if the framework did all the stuff it does and then gives an option to output a CloudFormation template or something could be used to port to another framework.

Anyway give it a shot and let me know what you think. You won’t regret it. If you want to see my completed todo app, check it out here: https://github.com/deeheber/amplify-todo.

The Secret Lives of Failed SQS Messages

Danielle Heberling — Wed, 18 Sep 2019 22:12:03 GMT

Photo by Samuel Zeller on Unsplash

A common pattern in serverless architecture is to have a queue before a function. This is great because you can create a second queue for all of the messages that failed in the function execution (or, if we want to put it in terms that don’t sound like we’re aggressively shaming them, we can classify them as having “encountered an error at some point”). This second queue is known as a “dead letter queue” or DLQ for short. If messages arrive in the DLQ, you can analyze what might have caused the error by reviewing any relevant logs, making changes to your stack, and running a redrive function to retry those messages. DLQs are useful for debugging your serverless application because they let you isolate problem-messages to determine why their processing isn’t working.

A redrive function is how we get those failed messages out of the DLQ and back into the original pipeline to retry the operation. I wanted to set up this type of architecture in a new stack using Stackery and decided to take a look around the internet for inspiration on different patterns and ideas on how to go about it. I found lots of information about dead letter queues, but no examples that demo-ed a redrive function to retry those failed messages. This post is meant to serve as a nice example for folks like me who want to do this on AWS and haven’t found many examples.

Update December 1, 2021 - AWS has added the ability to redrive your SQS messages in the console. See here for more information.

Update June 9, 2023 - AWS has added a new set of APIs to the AWS CLI for DLQ redrive. See here for more information.

The Setup

Happy Path

Let’s start out with our happy path flow. I’ve set up a stack that consists of an SNS topic > SQS queue > Lambda function. In Stackery it looks something like this:

Retry Path

In the event that something goes wrong during the function execution, I set up a second queue; this will serve as my DLQ for failed messages to be delivered. I then added a redrive function to allow me to retry those failed messages through the original pipeline. Here’s what that set up looks like:

I added a RedrivePolicy to the original SQS queue to point those failed messages to my DLQ by adding some <a href="https://aws.amazon.com/cloudformation/" target="_blank" rel="noopener noreferrer">Cloudformation</a> in my template.yaml:

Queue:
  Type: AWS::SQS::Queue
  Properties:
    QueueName: !Sub ${AWS::StackName}-Queue
    RedrivePolicy:
      deadLetterTargetArn: !GetAtt DeadLetterQueue.Arn
      maxReceiveCount: 3

By default, the queue’s message retention period is 4 days. I wanted to give myself some extra time to review the messages in the DLQ, so I upped the MessageRetentionPeriod on my DLQ to the max allowed 14 days like this in my template.yaml. This is the sci-fi beauty of my profession: creating more time is possible! As many before me have discussed on the floor of a dorm room, time is a malleable concept— and at least in software engineering, it has a practical purpose like gaining more days to review DLQ messages. Note that the values are in seconds:

DeadLetterQueue:
  Type: AWS::SQS::Queue
  Properties:
    QueueName: !Sub ${AWS::StackName}-DeadLetterQueue
    MessageRetentionPeriod: 1209600

Redrive Function Content

Now that the base architecture is all set up, let’s take a look inside the redrive function. This is how I decided to set mine up, but keep in mind that there are many different ways you can do this:

Here’s the high-level steps of what our retry function needs to do: Receive messages from the DLQ If no messages are received exit the function If there are received messages, loop through each message > send it to the original queue Delete the message from the DLQ on a successful send Repeat

Here’s what the code looks like written in nodeJS:

const aws = require("aws-sdk")
const sqs = new aws.SQS()

exports.handler = async (event) => {
  // Log the event argument for debugging and for use in local development.
  console.log(JSON.stringify(event, undefined, 2))

  while (true) {
    try {
      // Use long polling to avoid empty message responses
      const receiveParams = {
        QueueUrl: process.env.DLQ_URL,
        MaxNumberOfMessages: 10,
        WaitTimeSeconds: 1,
      }

      // Get messages from the DLQ
      // Continue looping until no more messages are left
      const DLQMessages = await sqs.receiveMessage(receiveParams).promise()

      if (!DLQMessages.Messages || DLQMessages.Messages.length === 0) {
        console.log(`NO MESSAGES FOUND IN ${process.env.DLQ_URL}`)
        // Exit the loop since there aren't any messages left
        break
      }

      console.log(`RECEIVED ${DLQMessages.Messages.length} MESSAGES`)

      for (const message of DLQMessages.Messages) {
        // Send message to original queue
        const outboundMessage = {
          MessageBody: message.Body,
          QueueUrl: process.env.QUEUE_URL,
        }

        console.log(`SENDING: ${JSON.stringify(outboundMessage, null, 2)}`)
        await sqs.sendMessage(outboundMessage).promise()
        console.log("SEND MESSAGE SUCCEEDED")

        // Delete message from DLQ
        const deleteParams = {
          QueueUrl: process.env.DLQ_URL,
          ReceiptHandle: message.ReceiptHandle,
        }

        console.log(`DELETING: ${JSON.stringify(deleteParams, null, 2)}`)
        await sqs.deleteMessage(deleteParams).promise()
        console.log("DELETE MESSAGE SUCCEEDED")
      }
    } catch (err) {
      console.log(`AN ERROR OCCURRED: ${err.message}`)
      console.log(JSON.stringify(err, null, 2))
      throw err
    }
  }
}

What to keep in mind when setting up a redrive function.

Receive message step

The default MaxNumberOfMessages to return is 1, so you’ll most likely want to up that number depending on your use case. I just went to the max allowed which is 10. If you hypothetically have 10 messages in your DLQ and do receiveMessage on that queue with a MaxNumberOfMessages set to 10, you might possibly not receive all of those messages in a single receiveMessage batch. It’s important to have looping to ensure that you receive all of the messages when they are available. Enabling long polling can help reduce the cost of SQS by reducing the number of empty responses. I decided to go this route by adding WaitTimeSeconds to my receiveMessage params.

Delete message step

Ensure the send message was successful before deleting it. I decided to wrap the entire function in a try, catch...so if there’s an error the function will stop processing.

As you can see, it does require a bit of additional setup, but creating dead letter queues and a redrive function to re-run failed messages can be extremely helpful in your application’s life cycle. You can go to https://github.com/deeheber/redrive to see an example of a simple stack that uses a DLQ and a redrive function in action.

For a solid and succinct description of the benefits of a DLQ (alongside lots of other info about them,) check out the AWS documentation <a href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html#sqs-dead-letter-queues-benefits" target="_blank" rel="noopener noreferrer">here</a>.

Note: This post was originally published on https://www.stackery.io/

Automated React Frontend Deploys

Danielle Heberling — Thu, 29 Aug 2019 22:12:03 GMT

As the frontend and backend are slowly merging closer together, more and more folks are writing full stack applications. While Serverless mostly is referring to the underlying infrastructure that tends to live a bit closer to the backend side, you can also use a serverless architecture to build and deploy the frontend portion of your application.

Note: Technology is always changing. I found what I feel is a "better" way to build/deploy a website to AWS. Check it out here

What We're Building

I decided to keep it simple/minimal in style as this is for personal use...so here’s how it ended up looking at the end.

Initial Setup

I started by adding an s3 bucket titled “FrontEnd” and enabled website hosting. In between my FrontEnd s3 bucket and my API gateway resource, I added the PopulateFrontend Lambda function to bridge the two resources together.

We’ll go more into detail on this later, but this Lambda function’s purpose is to build deploy the frontend part of the application everytime there is a deploy of the stack.

Also since we’re now going to be using a browser to make requests to the backend API, <a href="https://github.com/deeheber/text-to-speech-converter/blob/blog-post-2/template.yaml#L37" target="_blank" rel="noopener noreferrer">I enabled cors on my API gateway endpoints</a> and <a href="https://github.com/deeheber/text-to-speech-converter/blob/blog-post-2/src/ConvertToAudio/index.js#L65" target="_blank" rel="noopener noreferrer">ensured my backend Lambdas allowed cors in their headers on each return</a>.

Here’s how the visualization of the completed stack looks.

Populate Frontend - Custom Resource

There are a few ways you could go about automating the build/deploy of your frontend. After thinking through different options, I finally decided to use a <a href="https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources.html" target="_blank" rel="noopener noreferrer">CloudFormation Custom Resource</a>. The YAML for my custom resource in the template.yaml file looks like this:

PopulateFrontendDeployTrigger:
  Type: Custom::FunctionDeployTrigger
  Properties:
    ServiceToken: !GetAtt PopulateFrontend.Arn
    DeploymentTimestamp: !Ref DeploymentTimestamp

So when the Type property starts with Custom::, CloudFormation then looks at the ServiceToken field and sends a request to the resource specified on every deploy. In this case, the requests will go to my PopulateFrontend function. Anything after Custom:: can be whatever you'd like. Personally, I'd suggest naming it something that makes sense as to what it does...but that's totally up to you.

When using custom resources, it is very important to send back success and failure messages from the resource that was requested by CloudFormation. If you do not do this, then often times CloudFormation will hang for hours in a state that is difficult to exit. There are a few ways to go about doing this, but I decided to use the <a href="https://www.npmjs.com/package/cfn-custom-resource" target="_blank" rel="noopener noreferrer">cfn-custom-resource npm package</a>. Here is where I’m using this library to send my success or error message response back to CloudFormation: https://github.com/deeheber/text-to-speech-converter/blob/blog-post-2/src/PopulateFrontend/index.js#L25-L30.

Populate Frontend - Contents

Now that you know about custom resources and how they work, let’s dig a little deeper into what the PopulateFrontend function is doing and how.

So on every deploy this function is triggered via the request sent to the function ARN from our custom CloudFormation resource. When triggered it does the following to build and deploy our frontend.

Copies the frontend files to the Lambda /tmp folder
Grabs the API url from the function’s environment variable
Writes the API url to a config file (this is needed for the frontend to be able to talk to the backend)
Runs npm install and npm build
Npm build puts all of the built files in a folder called /build, so we then send all of those built files to our FrontEnd s3 bucket. Since the s3 bucket is enabled to static website hosting, it should “just work.”

Here's the final code for that function if you'd like to take a look: https://github.com/deeheber/text-to-speech-converter/blob/blog-post-2/src/PopulateFrontend/index.js

More on the Config File

I could’ve easily used another http request library such as <a href="https://www.npmjs.com/package/axios" target="_blank" rel="noopener noreferrer">Axios</a>, but since AWS Amplify has a <a href="https://www.npmjs.com/package/aws-amplify" target="_blank" rel="noopener noreferrer">nice library</a> that works well with sending http requests to API Gateway, I decided to go that route. Side plug...it also works well with GraphQL/AWS AppSync if http is not your preference.

In order to make calls to my API Gateway http endpoints via this framework, I need to feed it the API_URL in a config file, so I utilized the PopulateFrontend function to grab this url and write it into a config file on every deploy. It might be a tad bit overkill, but if I’m deploying to different environments I won’t need to think twice about this configuration...it’ll already be done for me.

As a note, if you want to work on the react frontend locally, you’ll need to add a config.js file in src/PopulateFrontend/frontend-content/src that looks something like this:

export default {
  backendAPI: "ENTER YOUR API GATEWAY URL HERE",
}

Closing

Hopefully this post gave you a nice example of how to implement an automated build/deploy of your frontend code with every stack deploy. You can see all of the code I referenced this this post here: https://github.com/deeheber/text-to-speech-converter/tree/blog-post-2.

If you want something more customized or complex, I’d recommend checking out AWS’s <a href="https://aws.amazon.com/codebuild/" target="_blank" rel="noopener noreferrer">code-build</a> and/or <a href="https://aws.amazon.com/codepipeline/" target="_blank" rel="noopener noreferrer">code-pipeline</a> services. If time permits, perhaps I’ll do another blog post or blog on those services.

Rekognition - Facing My Artificial Intelligence Fears

Danielle Heberling — Wed, 03 Jul 2019 22:12:03 GMT

Artificial intelligence (henceforth referred to as AI for the rest of this post) is becoming more and more prominent in our lives whether we realize it or not. This fact both excites and terrifies me as an individual. It is exciting because AI has the major potential to make all of our lives easier. It is terrifying because as a minority on multiple levels, I fear that my unique view and challenges in the world might not be reflected in those learning models if folks who look and act like me are not actively involved in the creation of those models. Furthermore, I’m also afraid for folks who don’t have as much privilege as I do.

So like the serverless developer that I am, I decided to scratch the surface by playing around a bit with the AWS service <a href="https://aws.amazon.com/rekognition/" target="_blank" rel="noopener noreferrer">Rekognition</a>. I wanted to see what I could do with it and also to test it’s accuracy.

Detect Labels

I began my journey with the detect labels API. To me this was super interesting, because all that I had to do was feed the API an image and the managed AWS service would do all of the heavy lifting and return a list of labels along with the level of confidence in that label. For example it might return something that would suggest “I’m 90% confident there is a dog” in this image.

Stackery and the AWS SDK made this possible for me to build. I set up an S3 Images bucket where uploading an image would trigger my ImageProcessor Lambda function. Then within the ImageProcessor Lambda function, I used the AWS SDK to make calls to Rekognition. For simplicity sake, I decided to just send the results to my Cloudwatch logs. Here’s what that architecture looked like.

If you'd like to see my code for this stack, you can check it out <a href="https://github.com/deeheber/face-match/tree/detectLabels" target="_blank" rel="noopener noreferrer">here</a>.

Compare Faces

Overall, the images that I uploaded were surprisingly accurate in the results returned, so I decided to go a step further. How about I create an application that when I upload an image of a person, it does a comparison through a collection of images of myself to see if there are any face matches? When I spoke about this idea with a coworker, he suggested I use this as a way to unlock a diary. Hardware on the diary takes a picture of me and then if there is a face match, the diary unlocks. Unsure if I’ll do that exact thing with this, but it was a pretty funny idea.

First, I needed to setup a collection that contained pictures of myself. These pictures are the ones that I will be comparing to, so it’s important to get a nice mix of pictures that are reflective of how I would look on an average day. So I opened up my terminal, installed the <a href="https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html" target="_blank" rel="noopener noreferrer">aws cli</a>, and ran:

aws rekognition create-collection --collection-id Danielle

I then set up a CollectionImages S3 bucket in which a put or delete event will trigger my ManageCollection Lambda function.

When I upload an image to the CollectionImages S3 bucket (put), the ManageCollection Lambda function adds the face to the collection via <a href="https://docs.aws.amazon.com/rekognition/latest/dg/API_IndexFaces.html" target="_blank" rel="noopener noreferrer">index faces</a>. Index faces will index all faces in an image, so for this case I only uploaded images that were me alone and set MaxFaces to 1. Upon successful index, I then stored the FaceId and bucket information in a DynamoDB table for later use.

When I delete an image from the CollectionImages S3 bucket, the ManageCollection Lambda function retrieves the FaceId that I stored in DynamoDB when uploading the image and uses that information to call <a href="https://docs.aws.amazon.com/rekognition/latest/dg/API_DeleteFaces.html" target="_blank" rel="noopener noreferrer">delete faces</a>. DeleteFaces removes that face from the collection. I then want to keep my data tidy, so I remove the information related to that deleted image from my DynamoDB table.

Now that my collection is all set, I have pictures to compare to...now let’s get to doing the face comparisons. To do this, I set up an Images S3 bucket. When I upload an image to that bucket, the put will trigger my CompareFaces Lambda function. The function then uses the <a href="https://docs.aws.amazon.com/rekognition/latest/dg/API_SearchFacesByImage.html" target="_blank" rel="noopener noreferrer">searchFacesByImage</a> method in the rekognition api to compare the face in the image I just uploaded to the images in my collection. This endpoint returns a lot of stuff, but I’m mostly interested in the list of face matches along with the confidence (in percentage). To keep things simple, I set it up to just log out the results to the CloudWatch logs for my CompareFaces Lambda function.

Here’s what that architecture looks like in Stackery:

You can also take a peek at my source code <a href="https://github.com/deeheber/face-match/tree/compareFaces" target="_blank" rel="noopener noreferrer">here</a>.

Circular Dependency Errors

If you take a look at my template.yaml in the detectLabels branch, you’ll see something really bad. <a href="https://github.com/deeheber/face-match/blob/detectLabels/template.yaml#L23" target="_blank" rel="noopener noreferrer">Here's</a> where I'm talking about.

I essentially gave my ImageProcessor function GetObject access to all S3 buckets in my entire AWS account. The best practice is to give the minimal permissions necessary…so I should’ve just given that function GetObject access to my Images S3 bucket.

The problem that I ran into when I did that was a circular dependency error when attempting to deploy this via Stackery...this specific error comes from CloudFormation. I had a function that responded to an event on the Images S3 bucket, but it also needed access to GetObject from that same S3 bucket. I had never run into this issue before and like the smart developer I am, I did a web search for that error and found this <a href="https://aws.amazon.com/blogs/infrastructure-and-automation/handling-circular-dependency-errors-in-aws-cloudformation/" target="_blank" rel="noopener noreferrer">wonderful post</a> that solved my issue.

For demo purposes, I did not update that in the detectLabels branch, but you can see that I fixed that <a href="https://github.com/deeheber/face-match/blob/compareFaces/template.yaml#L49" target="_blank" rel="noopener noreferrer">on this line</a> in the compareFaces branch. Since I already knew what my S3 bucket’s name will be, I place that in there instead of referencing the CloudFormation resource directly. So now, that function no longer has GetObject access to every single S3 bucket in my AWS account...it just has GetObject permission on that specific bucket without a circular dependency error.

Closing

Overall my testing has shown that AWS Rekognition is a lot more accurate than I had imagined. Despite this, I do think that it is important to ensure that new technology is representative of everyone especially if we plan to use AI to make decisions/judgements on things that have the potential to leave a large impact on an individual's life. This initial hands off experiment has gotten me more interested in exploring AI further and possibly learning how to create my own AI models in the future.

Building a Better Product through Customer Service

Danielle Heberling — Tue, 21 May 2019 22:12:03 GMT

As an individual coming from a customer support background that knows the value of being on the front lines, I am ecstatic that Stackery focuses on the customer across all teams. Moreover, I’m excited by the impact of having the front line perspective ingrained in the product we build. After all, the customers are the reason why we work so hard at what we do. In this post, I’m going to highlight some of the major benefits that I’ve noticed as a software engineer working in a periodic customer service rotation capacity. Here at Stackery, we call this the “Support Hero” role.

1. Better time management

The flow of work is very different when working on engineering tasks vs customer service tasks. The first thing that I noticed when working the customer support rotation was that I had to make sure I was in the office and available to receive customer inquiries from 9am-5pm every day that week. If I had to step out for a meeting or break, I had to keep an eye on our support channels and/or ask a teammate to cover for me while away. During my weeks of software engineering, I have a bit more freedom and flexibility when it comes to starting and ending my work day and can pretty much get up and take a break without needing to think about coverage. Beyond making sure I’m available if customers have questions, I also find myself needing to be flexible in order to balance having multiple conversations with different folks, oftentimes simultaneously. Since all inquiries are unique, they each require a different level of attention, care, and focus.

2. Clearer communication with others

A lot of the times companies have their own internal jargon of what they call things. This internal jargon often doesn’t map directly to what external customers are calling things. It’s easy to get into a habit of using a specific set of terms and phrases so much that you just assume everyone speaks the same way. When chatting with customers, I find that Stackery is not the exception to this and when I started my rotation, I often found myself asking for more clarification when a customer submitted an inquiry— I had to be certain we were on the same page. The more I speak with customers, the better I get at being able to intuitively read what they’re looking to do. This is a better way because the conversation can be more natural instead of a back-and-forth sequence of asking questions. A lot of times, I find answering the customer’s question to be quicker than that initial discovery of what they’re looking to do. It’s important to not jump ahead and skip that very important step at the beginning of the conversation.

3. A different perspective

I think many people in the modern workforce have worked at a company where the software engineers sit off in their own world and don’t get to hear the direct feedback from customers about the product they’re building. When creating a product and staring at the user interface for 40+ hours a week, it is easy to gain some heavy biases around how folks use the product and about what customers want. Through working in customer support, I’ve discovered a lot of bugs and things that seem intuitive to me, but in reality, are confusing to the bulk of the population. Being able to hear these concerns directly from a customer can help me attach a face and story to how this affects their daily work and gives me a higher urgency to ensure that reported issues are handled in a timely manner.

4. Increased product knowledge

An enterprise sized application is huge and has many different moving parts. It is impossible to fully master all of it. This is especially true if you’re a software engineer working on a product. In my personal experience, I’ve found myself to be an expert on the parts of the application that I had a key part in developing while I have significantly less knowledge about the other parts. Working in customer service, customers will ask questions related to every part of the application. I love this because it gives me more incentive to explore the other parts of the application that I might otherwise be unfamiliar with.

5. A better product. Period.

I want to give one example of how our customer support can improve the product. One of Stackery’s super powers is bundling and managing the configuration and environment variables so that all the settings you need to run in “staging” or “production” are in one place. We on the team knew that this feature helped users take the same set of functions and resources and deploy them in different environments. Our users, however, were more confused by these tools than empowered. We used that feedback to build something great.

Every week the “Support Hero” would hear from users confused by what these “environment” settings even did. We took their responses and spent some time redesigning the interface, and along with other feedback we designed a new “deploy” interface that clearly showed the environments you’d created, and let you choose to deploy your app to seperate environments in sequence.

our updated interface made the tool we’d already built a lot easier for our users to understand and use

This feedback, and the improvements, wouldn’t have happened if we didn’t have engineers doing support. The users weren’t asking for more environment support, it wasn’t a feature they were really aware of, so they weren’t going to tell Sales they wanted something they hadn’t heard of. And if all we’d looked at was user behavior, we probably would have assumed “people just don’t want to manage the same app in separate environments.” It was only by dealing with users directly, realizing that a tool we already had could help users if the User Interface (UI) was better.

With this process in place, and the product improvements driven by support hero experiences, we’ve seen our weekly active Stackery users triple. I beleive that a number of factors have influenced this including our the launch of <a href="https://www.stackery.io/blog/how-to-serverless-locally/" target="_blank" rel="noopener noreferrer">Stackery local invoke</a> but it also seems clear to me that the use of Stackery is growing because we are able to help trial users and inquiring minds more efficiently.

Acheive great things This type of process may not be appropriate for all organizations, but I thought I’d share a little bit about how our “Support Hero” philosophy has strengthened the Stackery product. I also believe that this has strengthened the Stackery software engineers on an individual level. I see my coworkers inspired to reflect on how our company is engaging with customers. It to makes the Stackery product and service better.

Want to benefit from a direct line of communication with Stackery’s serverless engineers? There are a variety of ways you can ask us questions, but one of the most fun is to join our team’s weekly livestream on Wednesdays at 10 AM PDT. Sometimes there are topics and guest hosts and sometimes we leave it as an open demo. Either way, it’s a great chance to get to know Team Stackery and ask us anything!

Note: This post was originally published on https://www.stackery.io/

Building a Text to Speech Converter with AWS Polly

Danielle Heberling — Mon, 06 May 2019 22:12:03 GMT

Recently, I spent some time building my own serverless text to speech converter and thought it might be helpful to share a bit about what I’ve learned during the process.

The Inspiration I have been studying to take some AWS certification exams. One of the most helpful resources for me in preparing for those exams has been the <a href="https://acloud.guru/" target="_blank" rel="noopener noreferrer">A Cloud Guru</a> courses. One of the lessons was a lab in which you build your own text to speech converter in Python for the purposes of being able to review the notes that you’ve taken during the course via audio.

After following along with the lab, I wondered what that application would look like if it were built using <a href="https://www.stackery.io/" target="_blank" rel="noopener noreferrer">Stackery</a> and NodeJS...so I decided to try it out on my own. Because I take a lot of public transit, I figured this application would be useful to take my notes in text form and convert into mp3 files that can be listened to while on the go.

The Architecture

I began by thinking through all of the high level steps and added/connected the resources in Stackery. There were a few iterations and many different ways to go about building this, but here’s the architecture I eventually settled on.

We have two API gateway endpoints: one to upload the text and convert it to an audio file and a second one to retrieve the metadata (including the s3 url) for each audio file.

The POST endpoint does the following: An API Gateway endpoint invokes the CreateNewFile Lambda function The CreateNewFile Lambda function adds metadata to a DynamoDB table and then hands the metadata info to the ConvertToAudio Lambda function The ConvertToAudio Lambda function separates the input text into batches of 2500-ish characters each (this is due to Polly size limits more detail can be found in the Future Improvements section of this post). Then for each batch it calls Polly to convert the text to audio and places the resulting audio stream in the /tmp directory for temporary storage. Once all batches are processed, the contents of /tmp are uploaded to s3 and the s3 url is added to the dynamoDB table.

Resource Permissions

Whenever you connect a resource (in this case Lambda functions) to a DynamoDB table or to an s3 bucket in Stackery visual mode, Stackery will grant that resource CRUD access to the target resource (in this case a DynamoDB table or an s3 bucket) since that is what most folks need. I enjoyed that setup for development purposes, but once the application was nearly complete, I only wanted to give my Lambda functions permissions to the actions they needed for security purposes.

So for example, my GetFile function allowed DynamoDB CRUD, but I just wanted it to have Scan and GetItem permissions to the table. Luckily with Stackery, I could use the template view to customize this beyond the default values. <a href="https://github.com/deeheber/text-to-speech-converter/blob/blog-post/template.yaml#L59" rel="noopener noreferrer" target="_blank">Here’s</a> what the end result looked like.

Another thing to note is if you want to give a function access to an entire bucket make sure you tack on /* to the end of the resource’s arn. It’s relatively simple to tack that onto the end of the resource’s arn via <a href="https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-join.html" rel="noopener noreferrer" target="_blank">CloudFormation’s join</a>. <a href="https://github.com/deeheber/text-to-speech-converter/blob/blog-post/template.yaml#L104" rel="noopener noreferrer" target="_blank">Here’s</a> an example of what that looks like.

Finally security wise, since Stackery doesn’t have a built in Polly resource (because Polly is not supported by CloudFormation), I also had to make sure I added permissions from my Lambda function to Polly like <a href="https://github.com/deeheber/text-to-speech-converter/blob/master/template.yaml#L119" rel="noopener noreferrer" target="_blank">this</a>.

Future Improvements

I view every one of my projects as a living breathing entity that is always growing and never fully complete, so here are some improvements I’d like to make as this application evolves. Most improvements are centered around how to better optimize the application in the case where I’d want to convert a massive amount of text (like an entire book) into an audio file.

Polly’s synthesizeSpeech API has a <a href="https://docs.aws.amazon.com/polly/latest/dg/limits.html" rel="noopener noreferrer" target="_blank">limit</a>. For now, I’ve built the app in a way where the input text gets chunked into blobs of about 2,500-ish characters then the program loops through each blob in order to call synthesizeSpeech for each blob to accommodate this limit. I may want to look into using the <a href="https://docs.aws.amazon.com/polly/latest/dg/asynchronous.html" rel="noopener noreferrer" target="_blank">SpeechSynthesis task API</a> instead.

It doesn’t affect smaller jobs or periodic blobs of notes, but if I want to scale this to accept an entire book of text there is a DynamoDB <a href="https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html" rel="noopener noreferrer" target="_blank">max item size limit</a> that might wreak havoc in the way the system is currently set up. I currently have it set to write all of the data including the entire text to be converted to audio to the DynamoDB table, I may want to truncate the amount of text that gets written to the metadata.

Again much like the Dynamo limit, there are also some <a href="https://docs.aws.amazon.com/lambda/latest/dg/limits.html" rel="noopener noreferrer" target="_blank">Lambda limits</a> that I may bump up against if I use this program to process an entire book of text. Notably, the storage size of the /tmp folder and the invocation payload limit since we’re using a Lambda to invoke another Lambda by <a href="https://github.com/deeheber/text-to-speech-converter/blob/blog-post/src/CreateNewFile/index.js#L45" rel="noopener noreferrer" target="_blank">sending over the needed data via the payload</a>. I may want to re-architect this in a way where we’re utilizing a queue to send that info over from one function to the other or possibly even combine the two functions into one. Not overly sure of a nice solution to the /tmp storage size issue at this moment, but I’ll get back to you on that.

Unrelated to scaling to allow for larger file processing, I’d like to build a frontend for this application along with a login/auth. For simplicity sake, I temporarily added the public-read ACL to the audio files that are uploaded to the FileStore s3 bucket. That would be a serious issue, if we wanted to ensure those files weren’t accessible to the public. For studying purposes, I don’t really mind, so this setup works for this particular use case. Closing This was a really fun and useful project, not only with getting the hands on experience exploring an AWS service that is new to me...but also by building something meaningful that I can use to continue to improve myself via studying for my AWS certification exams while on the go.

The full repository can be found <a href="https://github.com/deeheber/text-to-speech-converter" rel="noopener noreferrer" target="_blank">here</a>. Hope it helps inspire you to go out and build something that is both fun and meaningful.

Website Refresh With Gatsby

Danielle Heberling — Wed, 13 Mar 2019 22:12:03 GMT

Sometimes you just want a change. In this post, I am going to write about my experience rebuilding my website using Gatsby. All of the things that I say in this are from my own experience and it might not match yours. It is my hope that this might help you decide if Gatsby is the right choice for you or if you're already using Gatsby maybe get some ideas for your site.

Background

As a former employee of Squarespace, I get two free comped sites for life which is pretty awesome, considering that those hosting fees do add up quickly. With that said, my old website on Squarespace was great for what it was...but as I learned how to code, I wanted to customize it more and more beyond what any template offered.

As someone who used to support the Squarespace developer platform, I'm very aware that I could build my own template...but that has its own black holes. A lot of times the core platform CMS would update and things within custom templates built on the Squarespace developer platform would randomly break...I did not want to deal with this.

Requirements

I wanted something that was minimalist, accessible, looks great on various screen sizes, customizable, and easy to add more blog. I also wanted full access and ownership of all of the code. This way, I could have the freedom to easily move my entire site elsewhere if I so chose to.

Enter Gatsby

There is a lot of hype around <a href="https://www.gatsbyjs.org/" target="_blank" rel="noopener noreferrer">Gatsby</a> in the JavaScript community particularly, folks who use React. I didn't want to immediately jump on the bandwagon until it was a bit more mature. The more and more positive things that I saw being said about Gatsby on Twitter and viewing awesome Gatsby sites, I knew I had to at least give it a try...I'm very glad I did.

Gatsby Features I Think Are Cool

If you're new to React and/or just don't want to build everything completely from scratch there are many <a href="https://www.gatsbyjs.org/starters/?v=2" target="_blank" rel="noopeener noreferrer">starter templates</a> to choose from. These are created by Gatsby employees and members of the community. I think you can also start from scratch though if you want, but am not overly sure.

Gatsby also has great <a href="https://www.gatsbyjs.org/docs/" target="_blank" rel="noopeener noreferrer">documentation </a> and an active <a href="https://github.com/gatsbyjs/gatsby/issues" target="_blank" rel="noopeener noreferrer">GitHub repository</a> where you can file an issue to ask for help on something or file a bug.

I am personally not a huge fan of React router, so I was delighted when I learned that <a href="https://reach.tech/router" target="_blank" rel="noopeener noreferrer">Reach Router</a> comes with Gatsby. Major bonus is that it is highly accessible, since Reach Router was created with accessible navigation at its core.

Gatsby has a great plugin eecosystem. Some of these plugins are maintained by Gatsby while others are maintained by various open source developers. Here are a few that I like...though there are many more to explore:

<a href="https://www.gatsbyjs.org/packages/gatsby-plugin-react-helmet/" target="_blank" rel="noopener noreferrer">React Helmet </a>makes it a breeze to enhance the SEO of your website.
<a href="https://www.gatsbyjs.org/packages/gatsby-plugin-typography/" target="_blank" rel="noopener noreferrer">Typography </a>allows easy use of adding a prebuilt typography theme.
<a href="https://www.gatsbyjs.org/packages/gatsby-plugin-offline/" target="_blank" rel="noopener noreferrer">Offline </a>adds a service worker to your site to allow it work offline and when there are bad network connections.
<a href="https://www.gatsbyjs.org/packages/gatsby-plugin-feed/" target="_blank" rel="noopener noreferrer">Feed </a>creats an RSS feed or feeds for your site.

You can deploy a Gatsby site nearly anywhere you want. There are plugins to make deploying to an S3 bucket or on Netflify super easy. <a href="https://www.gatsbyjs.org/docs/deploying-and-hosting/" target="_blank" rel="noopeener noreferrer">The docs</a> have a list of ideas for hosting if you aren't sure with instructions on how to deploy to those different platforms.

Challenges I ran into

This one is mostly a first world problem, but the starter template that I decided to go with used some coding conventions that don't match my style, so I ended up spending some time doing things like getting rid of dangling commas, adding semicolons etc.

I think this may possibly have been due to the service worker, but I found that my site was very aggressibely cached. Lots of times when I made a change, it would not show on localhost until I did a hard refresh.

Not directly Gatsby related, but I attempted to use <a href="https://www.netlify.com/docs/form-handling/" target="_blank" rel="noopeener noreferrer">Netlify forms</a>, followed the docs, and still could not get the form the reliably submit. I filed an issue with Gatsby. They were amazing and replied super fast. We eventually decided the issue most likely was something on Netfliy's side. After more troubleshooting and talking with Netlify's support, I decided to give up on that since I wanted to ensure my form would reliably submit as I did not want to miss any messages and there are many other options out there to create a form.

The Future

I view pretty much every project of mine as a living, breathing entity that is never complete. I'm keeping track of ideas of things I'd like to implement in the future on <a href="https://github.com/deeheber/danielle-heberling-dot-xyz/issues" target="_blank" rel="noopeener noreferrer">GitHub</a>. Feel free to contribute if you'd like.

In addition to my issues listed on GitHub, I also think I'd like to deploy the site to an S3 bucket (it is currently hosted on Netlify) to make integration with other AWS services easier. For now, I think thinking I'd like to put that bucket behind a CloudFront distribution that can talk to a Lambda which processes the form and then sends me a customized email with the form submission info via SES and utilizes Cloud Monkey. Ok now I'm just making up fictional AWS services, I'll stop.

Final Thoughts

Working with Gatsby has been a joy. The community is so kind and helpful, and the docs are very detailed/searchable. Anytime I had a question 99% of the time, I could find it through a quick search since someone else had already asked the same question I was.

Gatsby is a great tool that can be used for various things such as a personal blog or a documentation site. In my opinion, it really shines with the various CMS options that you can connect to a Gatsby frontend. This is very helpful if you have people who don't know how to code or don't want to code updating or adding new blog blog or other content. They can add their content while the developer can work on customizing how the website or web app looks. I give Gatsby a 10/10...would use again.

Chaos Engineering Ideas For Serverless

Danielle Heberling — Thu, 24 Jan 2019 22:12:03 GMT

The <a href="http://principlesofchaos.org/" target="_blank" rel="noopener noreferrer">Principles</a> define chaos engineering as:

The discipline of experimenting on a distributed system in order to build confidence in the system’s capability to withstand turbulent conditions in production.

The high-level steps for implementing chaos experiments involve: defining your application’s steady state, hypothesizing the steady state in both the control and experimental groups, injecting realistic failures, observing the results, and making changes to your code base/infrastructure as necessary based off of the results.

Chaos experiments are not meant to replace unit and integration tests. They’re intended to work with those existing tests in order to assure the system is reliable. A great real-world analogy is that chaos experiments are like vaccines: A vaccine contains a small amount of the live virus that gets injected into the body in order to prompt the body to build up immunity to prevent illness. With chaos experiments, we’re injecting things like latency and errors into our application to see if the application handles them gracefully. If it does not, then we can adjust accordingly in order to prevent incidents from happening.

Sometimes chaos engineering gets a bad reputation as “breaking things for fun.” I believe the problem is that there’s too much emphasis on breaking things while the focus should be on why the experiments are being run. In order to minimize your blast radius, it’s recommended to begin with some experiments in non-production environments during the workday while everyone is around to monitor the system. On the people side of things, make sure you communicate with your entire team what you’re doing, so they aren’t caught by surprise. Once you have experience and confidence running experiments, you can then move onto <a href="https://www.stackery.io/product/deploy/" target="_blank" rel="noopener noreferrer">running them in production</a>. The end goal is to run experiments in production since it is difficult to have an environment that matches production exactly.

Traditionally chaos engineering at a high level is running experiments that often involve shutting off servers, but if you are in a serverless environment with managed servers this can pose a new challenge. Serverless environments typically have smaller units of deployment, but more of them. This means for someone who wants to run chaos experiments, there are more boundaries to harden around in your applications.

If you’re thinking about running some chaos experiments of your own in a serverless environment, some ideas of things to look out for are:

Performance/latency (most common)
Improperly tuned timeouts
Missing error handling
Missing fallbacks
Missing regional failover (if using multiple regions)

For serverless, the most common experiments involve latency injection or error injection into functions.

Some examples of errors you could inject are:

Errors common in your application
HTTP 5xx
Amazon DynamoDB throughput exceeded
Throttled AWS lambda invocations

A pretty neat trend I’ve seen is folks writing their own libraries to inject latency and/or errors into a Lambda function using the new Lambda layers feature. <a href="https://docs.stackery.io/docs/api/nodes/Function/#layers" target="_blank" rel="noopener noreferrer">Stackery makes it easy</a> to add layers to your function. Another idea is to implement chaos experiments as part of your CI/CD pipeline. If you don’t want to write your own library there are a lot of open source projects and also some companies that offer “chaos-as-a-service.”

If you’d like to go into more detail on implementation, I’d suggest checking out <a href="https://medium.com/@adhorn/injecting-chaos-to-aws-lambda-functions-using-lambda-layers-2963f996e0ba" target="_blank" rel="noopener noreferrer">this article</a> to see some code. This <a href="https://github.com/dastergon/awesome-chaos-engineering" target="_blank" rel="noopener noreferrer">GitHub repo</a> also has some great resources on the overall topic of Chaos Engineering. I hope this post gave you some ideas and inspiration on different ways you can test your serverless environment to ensure system reliability for your customers!

Note: This post was originally published on https://www.stackery.io/

Why I'm a Software Engineer

Danielle Heberling — Fri, 21 Dec 2018 22:12:03 GMT

As a software engineer from a non-traditional background, I’m often asked about how I got here and what motivated me to make a career change…this article will attempt to do a good job of explaining my path. Hope it helps you on your journey — whatever that journey might be.

The Early Years

Growing up in the early 1990’s the internet was still a pretty new idea to the general public. It would often take minutes for a simple HTML/CSS page to load and many folks didn’t have a PC in their house with access to the internet. Lots of people in my hometown would flock to the local library in order to use the internet. I was lucky enough that my father recognized that the internet was the future and invested in a home PC with dial up internet. I still remember when we first got the computer and he was just so excited to show us the internet for the first time.

The Musician Years

Fast forward a few years to university as a music education major. As a creative, I primarily surrounded myself with other people in various artistic disciplines. I was generally a more analog person as I saw technology to be robotic, calculated, and lacking the raw human emotion that went into creating good art.

This was until I had to take a music technology class as a requirement for my degree. In this class, we were taught how to use technologies like Finale, Garage Band, and Band in a Box. In this class, I learned that I could utilize technology to not only create my art, but to make it better. After completing that course, I often found myself researching and buying the next Apple product/upgrading it probably beyond what I actually needed just because I could.

The Tech Support Years

Fast forward even further. As an adult, I held various jobs, but eventually gravitated toward technical support. This was because I loved learning about new technology, enjoyed empowering/educating others, and didn’t know how to code (yet).

At this point in my career, I had been promoted enough where a major part of my job was acting as a bridge between engineering and customer support. We would have many cross-departmental meetings that involved activities such as going through open bug reports, discussing the overall customer impact of recent bugs/outages, and talking about common feature requests. Sadly, a lot of the time these conversations would be completely disregarded by the engineering team as if the meeting never happened.

I knew that my perspective was valuable and deserved to be taken seriously. I wanted the skills and knowledge required to be able to directly act on these requests from customers in order to create an overall better experience. I also wanted to bring human empathy into engineering and to instill a culture of respect for customer support from the engineering department. This is what ultimately led me into the journey of learning how to code.

TL;DR

Through my learning process, I realized that even though coding is very logical with many rules, there’s also an artistic aspect to creating an application since there are always multiple ways to approach and solve the same problem.

Anyway, shout out to all of those individuals/organizations who have both positively and negatively impacted my journey. It wasn’t easy or short, but I wouldn’t change any of it since I feel it gives me a unique perspective to bring to the table.

Stay tuned for the next part of my journey when I share about my discovery of the magic that is serverless…

Using AWS Amplify To Authenticate Users

Danielle Heberling — Wed, 24 Oct 2018 22:12:03 GMT

So you want to use AWS Cognito to authenticate users and have your user pool, identity pool, and app client all set up in the AWS console. ...the next question is how can you connect this with your React based frontend? While there are a few ways to go about doing this, this post is going to give you a brief overview on how to do this via a library called <a href="https://aws-amplify.github.io/" target="_blank" rel="noopener noreferrer">AWS-Amplify</a>.

AWS-Amplify is an open source project managed by AWS described as “a declarative JavaScript library for application development using cloud services.” I liked this particular library, because it has a client first approach and abstracts away some of <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/using-amazon-cognito-user-identity-pools-javascript-examples.html" target="_blank" rel="noopener noreferrer">the setup</a> required in the JavaScript SDK.

My favorite features of Amplify are: Authentication (via <a href="https://aws.amazon.com/cognito/" target="_blank" rel="noopener noreferrer">Cognito</a>), API (via API Gateway), and Storage (via S3), but this library has a lot more to offer than just those features. This post will focus on how to authenticate users from a React based frontend...more specifically user signup that has an email address verification step.

The Setup

First you’ll need to setup a config file to reference your already created AWS resources (in this case the user pool, identity pool, and client id) in your /src folder. The src/config.js file will look something like this :

export default {
   cognito: {
    REGION: ‘YOUR_COGNITO_REGION’,
    USER_POOL_ID: ‘YOUR_USER_POOL_ID’,
    APP_CLIENT_ID: ‘YOUR_APP_CLIENT_ID’,
    IDENTITY_POOL_ID: ‘YOUR_IDENTITY_POOL_ID’
  }
};

Then in your src/index.js file where you setup your react app, you’ll need to configure aws Amplify. It’ll look similar to this:

import React from ‘react’;
import ReactDOM from ‘react-dom’;
import Amplify from ‘aws-amplify’;

import config from ‘./config’;
import App from ‘./App’;

Amplify.configure({
  Auth: {
    mandatorySignIn: true,
    region: config.cognito.REGION,
    userPoolId: config.cognito.USER_POOL_ID,
    identityPoolId: config.cognito.IDENTITY_POOL_ID,
    userPoolWebClientId: config.cognito.APP_CLIENT_ID
  }
});

ReactDOM.render(
  <Router>
    <App />
  </Router>,
  document.getElementById(‘root’)
);

The mandatorySignIn property is optional, but is a good idea if you are using other AWS resources via Amplify and want to enforce user authentication before accessing those resources.

Also note that for now having a separate config file might seem a bit overkill, but once you add in multiple resources (i.e. Storage, API, Pub Sub etc.) you’ll want that extra config file to keep things easy to manage.

Implementation Overview

The signup flow will look like this:

The user submits what they’ll use for login credentials (in this case email and password) via a signup form and a second form to type in a confirmation code will appear.
Behind the scenes the Amplify library will sign the user up in Cognito.
Cognito will send a confirmation code email to the user’s signup email address to verify that the email address is real.
The user will check their email > get the code > type the code into the confirmation form.
On submit, Amplify will send the information to Cognito which then confirms the signup. On successful confirmation, Amplify will sign the user into the application.

Implementation Part 1

First in your signup form component, you’ll need to import Auth from the Amplify library like this:

import { Auth } from ‘aws-amplify’;

As you create your form, I’d suggest using local component state to store the form data. It’ll look like your typical form with the difference being using the Amplify methods in your handleSubmit function whenever the user submits the form. The handleSubmit function will look like this:

 handleSubmit = async event => {
    event.preventDefault();

    try {
      const newUser = await Auth.signUp({
        username: this.state.email,
        password: this.state.password
      });
      this.setState({
        newUser
      });
    } catch (event) {
      if (event.code === ‘UsernameExistsException’) {
        const tryAgain = await Auth.resendSignUp(this.state.email);
        this.setState({
          newUser: tryAgain
        });
      } else {
        alert(event.message);
      }
    }
  }

On success, Amplify returns a user object after the signUp method is called, so I’ve decided to store this object in my component local state so the component knows which form to render (the signup or the confirmation).

Before we continue let’s go over a quick edge case. So if our user refreshes the page when on the confirmation form and then tries to sign up again with the same email address, they’ll receive an error that the user already exists and will need to signup with a different email address. The catch block demonstrates one way of handling that possibility by resending the signup code to the user if that email is already present in Cognito. This will allow the user to continue using the same email address should they refresh the page or leave the site before entering the confirmation code.

Implementation Part 2

So now the user is looking at the confirmation form and has their confirmation code to type in. We’ll need to render the confirmation form. Similar to the signup form it’ll look like a typical form with the exception being the function that is called whenever the user submits the confirmation form. The handleSubmit function for the confirmation form will look similar to this when using Amplify:

handleConfirmationSubmit = async (event) => {
  event.preventDefault()

  try {
    await Auth.confirmSignUp(this.state.email, this.state.confirmationCode)
    await Auth.signIn(this.state.email, this.state.password)

    this.props.isAuthenticated(true)
    this.props.history.push("/")
  } catch (event) {
    alert(event.message)
  }
}

So it is taking in the form data, using Amplify to confirm the user’s email address via the conformation code and signing in the user if successful. You can then verify if a user is signed in via props at the route level if you’d like. In this case, I arbitrarily named it isAuthenticated and redirected the user to the root path.

The complete docs for using the Auth feature of Amplify can be <a href="https://aws-amplify.github.io/docs/js/authentication" target="_blank" rel="noopener noreferrer">found here</a>. We’ve only scratched the surface in this post, so go forth and explore the all of the different features that Amplify has to offer. I’ve found it has a very nice declarative syntax and is very readable for folks who are new to a codebase. For building further on your React-based serverless applications, I highly recommend Stackery for <a href="https://www.stackery.io/product/" target="_blank" rel="noopener noreferrer">managing all of your serverless infrastructure</a> backed up by seamless, git-based version control.

Note: This post was originally published on https://www.stackery.io/

How Being a Musician Has Made Me a Better Software Engineer

Danielle Heberling — Mon, 06 Aug 2018 22:12:03 GMT

Oftentimes in society, we like to categorize things in two opposing binaries. Black and white. Dog people and cat people. Creative and logical. Artistic and mathematical. What about the gray area? What if something doesn’t fit neatly into this box or that box and instead loosely fits into both boxes?

I knew from a relatively young age that I wanted to be a musician. Music has always been a major influence in my life whether that be through my own medium of expression or through the inspiration that I receive from consuming music from other creators. What I didn’t know at that young age was that the skills that I learned as a musician though seemingly unrelated translate very well into other topic areas.

Since I made a career switch from being a music teacher to being a software engineer, here’s an overview of how the skills I learned as a musician have made me a better software engineer.

Teamwork

Story time: when I was a music student in university there were two woodwind quintet groups. Group #1 was all of the individuals who were first chair in the entire university for their respective instrument. Group #2 (my group) were individuals that were good acquaintances and all in the same graduating class.

After a concert that showcased both groups, we all received feedback that group #2 outperformed group #1. The reason…well it wasn’t because each individual in the quintet was the “best” at our respective instrument…it was because we did a great job of collaborating with each other and blending to form a cohesive ensemble.

I feel this can also translate to a software team. Say for example you have a group of “rockstar” programmers…it doesn’t automatically mean that they as individuals are also great at collaborating or working on a team (disclaimer that some “rockstars” can). The best software teams that I have been on didn’t necessarily have the best individual programmers, but they were group of individuals that had a primary focus of helping each other out with the end goal of making the finished product better accompanied by a curiosity to learn something new.

Practice

As a musician, I spent about 90% of my time practicing and 10% performing. Even more time was spent making my own oboe reeds…but that’s probably a separate blog post.

The same can be applied to software development. Currently as a professional software engineer, I spend the bulk of my time researching a new framework or technology, building throwaway apps, and reading documentation. The least amount of time spent is writing code that ends up getting shipped into production

Attention to detail

In most academic settings a “B” grade is good enough to pass. Different institutions define a “B” differently; however, most places define is as 80–90% correct answers. Now think of that in terms of music…if I played a song with 90% of the notes being accurate…it would sound completely terrible.

The same principle applies to building software. How many times have you spent 30+ minutes trying to figure out why a linter was yelling at you…only to find you were missing a semi-colon or a closing curly brace? Near perfection is required.

Learning a new language

When you think of a spoken language, there are always smaller building blocks that come together to build something larger. For example, a letter…becomes a word…becomes a sentence…becomes a paragraph…becomes a novel.

It’s similar in music. A note becomes a scale…sometimes multiple notes are played at the same time which creates a chord…multiple notes, scales, and chords come together to create a song.

In software development we have multiple programming languages and most languages have common building blocks that come together to create a script or software product. Those common building blocks include variables, functions, and conditional expressions.

Context Switching

This next part is my personal opinion. As a musician I felt that the people that were the most fun to perform with were the folks who could just walk into a room and almost instantaneously fit in…both on a musicianship level and on a social skills level. I always enjoyed playing with the people that played multiple instruments and genres of music.

On the other hand, my favorite software engineers I’ve worked with are very good at reading the room and can adjust how they interact to fit the team’s preferred communication/working style. It has also been helpful (but not mandatory) to work with software engineers that know multiple programming languages and have worked on many different codebases of various sizes.

Conclusion

If you’re already a software engineer and reading this; hopefully, you feel inspired to try performing or creating music. Not only is it a lot of fun, it will make you a better software engineer.

If you’re a musician, in Portland Oregon, and want to dabble with coding there are some free beginner learn how to code workshops on Meetup.

What are some other parallels that I might have missed? Also feel free to share how skills learned in other disciplines have helped you become a better software engineer and why in the comments.