Please provide feedback in https://discord.com/invite/adh94jZH37 on which dimension(s) that you want to scorer to be improved on. We should be able to incorporate these improvements on the generic scorer before going into training a specific scorer for domain specific use-cases.
great looking ui and implementation, and kudos on the launch. how do you handle cases where predefined scoring metrics don’t fully capture what ‘good’ means for a specific use case, like ranking legal documents or detecting nuanced sentiment in customer reviews?
hey will, the metrics in the scoring system are not predefined but rather generated for a use case by breaking down a subjective set of conditions into a tree of metrics that combine various objective metrics into a subjective aggregate. Here is a prefilled playground with dimensions for the sentiment analysis in customer reviews use case. You can see how it breaks down and if you put a review and click Run you should see the scores combine from individual custom dimensions to the aggregate score. Hope this helps!
I like how there is a UI I can interact with before getting started with the SDK, helps me understand how the platform works.
What kind of fine-tuning capabilities does the user have on the scorer? Would I be able focus on improving certain scorer dimensions over others?
https://build.withpi.ai/ should work you through the platform capability.
Please provide feedback in https://discord.com/invite/adh94jZH37 on which dimension(s) that you want to scorer to be improved on. We should be able to incorporate these improvements on the generic scorer before going into training a specific scorer for domain specific use-cases.
great looking ui and implementation, and kudos on the launch. how do you handle cases where predefined scoring metrics don’t fully capture what ‘good’ means for a specific use case, like ranking legal documents or detecting nuanced sentiment in customer reviews?
hey will, the metrics in the scoring system are not predefined but rather generated for a use case by breaking down a subjective set of conditions into a tree of metrics that combine various objective metrics into a subjective aggregate. Here is a prefilled playground with dimensions for the sentiment analysis in customer reviews use case. You can see how it breaks down and if you put a review and click Run you should see the scores combine from individual custom dimensions to the aggregate score. Hope this helps!
https://build.withpi.ai/shared/021c9990-c02a-477c-8e32-fd2d5...
Really nice ui aesthetic - inspiration from PostHog?
lol wow nice catch
posthog design is great- you should be proud. Keep it up
super cool!