The Rise of Super Models

With the recent launch of Meta's Segment Anything Model we see a new image super model that can segment photos into every object on the screen. Even when there are 50+ objects that start to blend together, the model can still segment them out. The craziest part is also the speed at which this can be done. It is so fast that it is being used for videos already. This is a huge step forward for computer vision!

In the photo below you can see the model segment out VCRs, TVs, speakers, even the shelf. It feels like a super model that can do everything.

Super Models are Still Limited

While there are tons of examples on their website, and you can play with it yourself, it is still not perfect. A common use case for developers would be to use this model within the context of a webpage. Lets take the New York Times homepage:

With this model, you can see that it misses large chunks the page, including article titles, date, and even menu items. This is a huge limitation for developers wanting to use this model within their automation framework.

Specialized Models are still valuable

Despite being less generalizable, specialized models are still going to be very valuable for developers. While super models are great, and tremendous steps forward in the field, they are not going to be able to handle every use case. This is where specialized models come in. They are able to handle the edge cases that these super models can't.

The New York Times homepage example before, using a specialized modelm trained by Dev Tools AI, on hundreds of apps & webpages, can segment all the core elements:

With a specialized model it is able to get all the objects on the page, and in the images. It can also detect what are buttons like the search icon & menu. This can be used both for automation, and for accessibility in apps where there isn't accesibility inherently (lack of alt tags, inaccesible doms for screen readers, etc).

Conclusion

While this is just one example in the computer vision field, this will apply to all of the new super models. These super models are good at many things at once, but are ultimately overly generic. There will be a specialized model that can out perform GPT on medical or legal questions for example, but will be at the cost of not being able to answer questions on art history as well. Specialized models, for specific domains, can be applied when needed & will be the key to unlocking the full potential of AI.

Dev Tools AI will continue to lead innovation in computer vision for Web & Mobile apps, and only continue to improve. To try it out, sign up today!

Re-Think the DevOps Cycle​

When we started Dev Tools AI, we believed that AI would change the way we build software and touch everything in the Dev Ops cycle. We are now seeing this happen in real-time. This has been incredibly exciting to see, and be apart of. We are starting to see the first wave of AI benefits in the DevOps cycle, and it isn't just the 30% of code written being written by AI. There are new methods, like Stacking that are allowing developers to give higher level input and let the AI write the code.

Scalability without Headcount​

With these ever improving AI systems, we are able to scale our development efforts to be much larger, but not through hiring more engineers. This is a huge bennefit for companies, and employees able to utilize these new tools. Developers can now write more features, faster, and with higher quality. This is a win-win for everyone.

Going past code assistant​

It isn't just writing code that is going to be affected by AI. We are seeing AI being used to help with debugging, testing, and even doing deployments + monitoring. This is just the beginning, and we are excited to see what the future holds...

Quality Control will be Paramount​

As AI takes on larger and larger roles in the development lifecyle, the most important role for humans to play will be ensuring the quality of the end product. If a human isn't there to do the final review of the output, then the quality will suffer. The human is the oracle that can ensure the AI is doing what it is supposed to be doing.

The Future is Now​

When we started Dev Tools AI, we believed that AI would change the way we build software and touch everything in the Dev Ops cycle. We are now seeing this happen in real-time. This has been incredibly exciting to see, and motivated us to expand past just helping developers write better UI utomation code, and into the entire Dev Ops cycle.

Launch of Reviewify - AI Powered Code Reviews​

Our first step in the road is the launch of Reviewify, an AI powered code review tool. Reviewify is a tool that helps developers write better code, faster. We leverage the power of GPT-4 and linters to in-line comments on every review within minutes to help give feedback to the developers in as close to real time as possible.

Lessons Learned​

Leverage AI Everywhere​

While building Reviewify, we leveraged the power of AI everywhere. We used DALLE-2 to generate the Reviewify logo (and all our blog post images). We used chatGPT to help with the copy on the page, and especially with the code. ChatGPT was especially awesome at helping us understand APIs as we added support for GitHub & GitLab.

Prompts Are King​

The way you structure the prompt has such a massive impact on the results you get back. When we first started experimenting with GPT we had simple prompts like Review the following code. This would give wild results back that were impossible to process.

We iterated over and over, including edge case and boundry tests for when certain conditions happen like finding no issues. This greatly simplified our post-processing needed to ultimately post a comment on a code review.

AI is Finiky​

The funniest thing that we have learned is how sensitive the system can be. For a code block with an error, lets say:

print("Starting script")
for i in range(10):
    print(i)
    for j in range(10):
        print(i / j)
print("This line will never be reached")

The different messages the AI system will return changes depending on if a new-line is present at the end or not. Despite the code being the exact same, something as small as that can have differing impacts.

What's Next​

This is just the first step, we aim to have many more products in the coming months to help development teams work more efficiently!

We think the pace of development will only continue to increase and we want to not just be apart of history, but to guide it. Stay tuned to see what's next!