Where do we draw the line? On building tools and unintended consequences
I’ve been having trouble sleeping lately because of a question I can’t shake: If something I built is being used to cause harm, am I responsible for that harm?
I realize that’s a heavy way to start a blog post. But I think a lot of us in tech are sitting with variations of this question, and we’re not talking about it enough. So let’s talk about it.
Why I got into this
I didn’t get into technology to get rich or because I thought it would be easy. I got into it because I’m a problem solver, and I genuinely wanted to make things better. I wanted to build tools that help people do their jobs more easily, that reduce friction, that make someone’s day a little less frustrating.
I think a lot of us started with some version of that. We believed - maybe still believe - that technology can be a force for good. If we build the right things in the right ways, we can actually improve people’s lives.
And then reality gets complicated.
The moment of realization
Recently, I found out that an open source tool I created is being used by an organization whose work I believe causes real harm in the world.
I won’t go into specifics - that’s not the point of this post. But when I found out, it hit me hard. This was something I built with good intentions. I released it to help people. I wanted to contribute something useful to the community. And now a company I fundamentally oppose is benefiting from my work.
Here’s the thing about building tools: once they’re out in the world, you don’t get to control who uses them.
If you release something open source, that’s kind of the whole point. You’re giving it away. Anyone can use it for anything. A small nonprofit trying to manage their donated equipment can use it. A school district can use it. And yes, a company whose work you find morally reprehensible can use it too.
If you work on enterprise software, you serve whoever pays for it. You don’t get to approve each customer based on whether you agree with their mission. The company needs revenue to survive. The salespeople closed the deal. And now you’re supporting them, whether you want to or not.
This is just how it works. Except… is that okay?
The line we don’t talk about
I keep thinking about this metaphor: imagine you’re a tool maker, and you build a hammer. You’re proud of it. It’s well crafted, it does exactly what it’s supposed to do. You sell it or give it away, and someone uses it to build houses for people who need them. Great! That’s what you wanted.
But then you find out someone else is using your hammer to break windows. Or worse.
Are you responsible for that? You didn’t tell them to do it. You didn’t even know they’d bought it. You just built a hammer. Hammers are neutral tools, right?
Except we’re not building hammers. We’re building surveillance infrastructure, data processing systems, automation tools, communication platforms. The things we build are being used to make real decisions about real people’s lives. And some of those decisions cause real harm.
The tension
This is where I get stuck. I keep cycling through three positions:
Position 1: “I’m not responsible for how people use my tools.”
This is the libertarian tech approach. You build something, you release it, and what happens after that is not your problem. People have agency. They make their own choices. You can’t control the whole world.
This feels… insufficient? Like, if I knowingly give someone a weapon and they hurt someone with it, I don’t get to just shrug and say “not my problem.” At some point, “I didn’t know” becomes “I didn’t want to know.”
Position 2: “I’m completely responsible and should stop building anything that could be misused.”
This is the other extreme. If your code could potentially be used for harm, you shouldn’t write it. If your company serves customers you disagree with, you should quit.
But this also feels untenable. Almost any tool can be misused. And most of us need jobs. Refusing to work anywhere that has any problematic customers means… refusing to work almost anywhere? Plus, stepping away just means someone else builds it who maybe cares even less about the ethics.
Position 3: “It’s complicated and I have to figure out my own line.”
This is where I land most days, but it’s deeply uncomfortable because it means I have to actually think about each situation. I can’t just follow a rule. I have to ask:
- What is this tool actually enabling?
- Who’s using it and for what?
- Is the harm direct or indirect?
- What’s the alternative if I don’t build/maintain this?
- Can I live with my level of involvement?
The open source dilemma
Open source makes this even more complicated. The whole ethos of open source is about freedom - freedom to use, modify, distribute. You don’t put restrictions on who can use it because that undermines the entire point.
But what if someone you fundamentally oppose uses your code? What if your contribution to the commons ends up being used to surveil people, or enforce policies you find unjust, or enable systems you’d actively work against if you could?
You can’t un-release it. You can archive it and stop maintaining it, but they’ll just fork it. You can add a strongly worded README about your values, but that’s purely symbolic.
And even if you could add license restrictions (which gets legally messy fast), who decides what’s “ethical use”? Your ethics aren’t universal. Someone else might think you’re the problematic one.
The enterprise employee dilemma
If you work for a company that builds enterprise software, you probably don’t get to choose your customers. The sales team closes deals. Your job is to support those customers, regardless of how you feel about them.
Maybe your company serves hospitals and schools and nonprofits doing important work. Great! But they also serve… other organizations. Organizations whose work you might find anywhere from “ethically questionable” to “actively harmful.”
So what do you do? Do you:
- Compartmentalize it as “this is my job, not my personal choices”?
- Try to advocate internally for ethical customer guidelines?
- Ask to be removed from supporting specific customers?
- Leave for a company with a customer base you’re more comfortable with?
- Accept that no company is perfect and this is just part of working in tech?
There’s no easy answer. And the smaller the company, the fewer options you have. “We can’t afford to turn away revenue” is a real constraint, especially for startups. But it’s also how companies end up doing things they later regret.
Where is the line?
I keep asking myself: if the harm was more obvious, more direct, would I have the courage to speak up? What if I was building tools being used for large-scale human rights violations? What if my code was part of separating families, tracking people fleeing violence, or targeting marginalized groups of people? At what point does “I need this job” or “I’m just one person” stop being enough?
I think the question isn’t “is there a line” but “where is YOUR line?”
For some people, the line is literal violence. If your code directly enables physical harm, that’s a no-go.
For others, it’s about intent. If a tool is being used for something it wasn’t designed for, that’s different than building something explicitly for a harmful purpose.
For others still, it’s about degree of separation. Directly building surveillance software is different than building a database that someone else uses in their surveillance software.
And for some people, there is no line. Everything is neutral. It’s all just tools.
I don’t know where my line is yet. I’m trying to figure it out. All I know for sure is that I have one.
The Rationalization That Tempts Us
Here’s a rationalization I hear a lot - and that I find myself considering: “If you quit over something like this, they’ll just hire someone else. Your individual protest won’t change anything. The organization you’re uncomfortable with will keep doing what they do regardless. At least if you stay, you’re employed. At least the company can take that revenue and use it to serve customers you do support. At least you might be able to do some good from within the system.”
And you know what? There’s some truth to this logic. Individual actions often feel futile. One person leaving a job rarely changes corporate behavior. One company refusing a customer just means that customer goes elsewhere.
But here’s what bothers me about this reasoning: you can use it to justify almost anything.
“If I don’t build this, someone else will” is how surveillance tools get built. It’s how weapons get manufactured. It’s how every ethically questionable technology ends up in the world. At some point, “someone else will do it anyway” stops being a reason and becomes an excuse.
And the “at least we can take their money to do good” part? That’s just sanitizing a moral compromise. You’re still enabling the harm. You’re just trying to balance it out with good things, and hoping the math works out in your favor.
This rationalization is seductive because it contains truth. But I’m not sure it’s sufficient justification for participation in harm. Or maybe it is, and I just haven’t figured out when it applies and when it doesn’t.
What I’m sitting with
Here are the questions I keep coming back to:
If I see harm happening and I have the ability to speak up, and I don’t - what does that make me?
Not speaking up because I’m scared of consequences is human. But at some point, does fear become an excuse for complicity?
If I’m benefiting from a system that causes harm, even if I’m not directly causing it myself, what’s my responsibility?
I’m getting paid. The company I work for is getting paid by customers who might be doing harmful things. At what point does “I need to pay rent” stop being a sufficient justification?
How do I balance my ideals with the reality of needing to survive in capitalism?
This is maybe the most frustrating one. I can have all the ethics in the world, but if I can’t pay my bills, what good are they? And yet, “I need money” can become a justification for almost anything if you let it.
Is there a difference between building something anyone can use versus choosing to support specific users?
My gut says yes, but I can’t fully articulate why. There’s something different about releasing a tool to the commons versus actively providing customer support to an organization you find harmful. But where exactly is that line?
I don’t have answers
I want to be really clear about this: I don’t know what the right thing to do is. I’m not writing this to tell you what you should do. I’m writing it because I’m genuinely struggling with these questions, and I suspect I’m not alone.
Maybe you’re a developer who just found out your code is being used in ways you never intended.
Maybe you’re working for a company that serves customers you’re uncomfortable with.
Maybe you released something open source years ago and just discovered it’s part of someone’s harmful infrastructure.
Maybe you’re trying to figure out whether to take a job at a company whose mission you believe in but whose customer base includes organizations you don’t.
I think we need to talk about this more. Not in a preachy, “here’s what ethics demands” way, but in an honest, “this is messy and complicated and I don’t have it figured out” way.
What helps?
I’m curious how other people navigate this. What helps you think through these decisions?
Do you have hard lines you won’t cross? How did you figure out what those are?
Have you ever left a job over ethical concerns? Did you regret it, or was it the right call?
Have you found ways to advocate for change within a company, or does that feel futile?
Do you think it’s even possible to work in tech without some level of ethical compromise, or am I being too pessimistic?
I don’t know if there are “right” answers to any of this. But I think the conversation itself matters. Because the alternative - just not thinking about it, just building whatever we’re asked to build and collecting our paychecks and telling ourselves it’s not our problem - that feels worse.
At least if we’re struggling with it, we’re paying attention.