AI Drop: Social Compliance Generator
Prove to us all that you're not a cereal rapist.
This is Week 1 of Code & Context’s AI Drop of the Week. Read the series introduction to learn more about the project, and make sure you subscribe get new AI Drops in your inbox. Paid subscribers get exclusive access to use the tools we build!
People who don’t post on social media are our worst people, probably. If you’re not a criminal pervert, if you aren’t subverting Taylor Swift or The New Yorker, why wouldn’t you post? If you haven’t danced online in a musical video short, can we afford not to cancel you?
But even the best of us sometimes struggle to get it right, right? If you’re like me, one of those that can’t organize well enough; one that hawks tuah on the decency of our commons; one who can’t even–dammit–be bothered to post one beautiful thought, select one clever image, or compose a devastating song every day for your comrades: this week’s AI Drop will help you.
The Brief
Every AI Drop starts with The Brief, my 30 second initial description of the problem and how I think the solution will go. Very often, as it happened this week, the tools and tech I use to reach the solution look very different from the ones sketched out here:
Create an “I hate social media social compliance generator”. Pick a random news topic, feeling, or x of the month and get a generated social media text, song, image and video. Auto posting to my Twitter feed and available to the public by June 10. Sample post: it’s Nepalese Cat Month! With a musical theme song generated by Sono and an image generated by Dall-e.
APIS to use:
Posting: Facebook
Text generation: Gemini Flash
Song generation: Sono, audio, or elevenlabs
Image generation: stability or Dall-e
Generate a new post every day
Create on a website which allows user to enter a prompt (or get a random prompt) and generate the post that can be copied to paste into their social feed.
Spoiler Alert: The Result | The Code
The Buildout
The project implementation involved several key components, each chosen to address specific needs and explore new technologies.
For scheduling, I opted for Trigger.dev V3. I'd had an invitation sitting in my inbox for months, and this seemed like the perfect opportunity to give it a try. It also offered a chance to work with Node.js and TypeScript, a departure from my usual Python-based projects.
Content generation relied on a trio of AI models:
Claude Haiku from Anthropic for text generation. I've always appreciated Claude's writing quality, and it didn't disappoint here. It not only generates the main text for posts but also creates prompts for image and music generation.
DALL-E 3 from OpenAI for image creation, working off the prompts provided by Claude.
Facebook’s Musicgen model for audio content, which I ran locally using the transformers.js library from HuggingFace. This was a bit of an adventure, as you'll see later.
For the social media platform, I initially attempted to use Facebook. However, the complexities of Meta's Business Suite proved to be more than I bargained for. So, I pivoted to X.com (formerly Twitter), which is where most of AI happens anyway.
The project also incorporates Supabase for content storage and database operations, and FFMPEG for combining generated images and audio into video content. Each of these components added its own layer of complexity to the project.
Input from the GPT Committee
GPT-4 played a bit part in this project, giving me an initial skeleton script by adapting a Python Twitter bot tutorial from the X.com developer site to work with TypeScript.
Surprises, Caveats, and Takeaways
Posting to social media via API proved to be significantly more complex than anticipated. Each platform has its own set of authentication requirements and API quirks, making this a more time-consuming process than expected.
The JavaScript support for HuggingFace models is not yet on par with their Python counterparts. I had to fork the transformers library to get Musicgen working properly in a JavaScript environment. It was a bit like trying to fit a square peg in a round hole, but we got there in the end.
Integrating FFMPEG for video creation was more challenging than expected, requiring careful configuration and error handling. Let's just say FFMPEG and I now have a complicated relationship.
The initial attempt with Facebook highlighted the importance of considering platform-specific requirements and limitations when designing cross-platform bots. It was a humbling reminder that not all platforms are created equal.
Performance can vary quite a bit. The generation time (currently about 40 seconds on Trigger.dev) is mostly linked to loading the Musicgen model and running inference against Trigger.dev’s CPUs.
Better for Next Time
Implement a system to generate content based on current headlines or specific newsfeeds, making the posts more relevant and engaging. Right now, it's not exactly breaking news.
Explore more advanced models or fine-tuning techniques to enhance the quality and relevance of generated images and music.
Work on reducing the generation time, possibly by optimizing API calls or exploring more efficient models.
Extend the bot to support multiple social media platforms simultaneously, starting with the planned Facebook and Instagram integrations. Why stop at annoying people on just one platform?
Develop a user interface that allows individuals to customize the bot's behavior, such as setting preferred topics or adjusting posting frequency.
Implement a content moderation layer to ensure all generated content adheres to platform guidelines and ethical standards. We're aiming for social compliance, not social exile.
Add functionality to track engagement metrics and use this data to refine the content generation process over time. Because if we're going to create an AI influencer, we might as well know if it's actually influencing anyone.
This project, while functional, is very much a work in progress. It's an exciting glimpse into the potential of AI-assisted content creation, but there's still a lot of room for refinement and improvement. Stay tuned for the next AI Drop - who knows what digital adventures await!
The Artifact
The bot has been running flawlessly since it launched in early June. You can see the outputs on X.com here: Social Compliance Bot.
You can fork and adapt the code yourself: Social Compliance Generator on GitHub.
Paid subscribers can try the web version of the social compliance generator, and generate content for your own social compliance needs.
[LINK BUTTON TO PAID SUBSCRIBER VERSION]