@fluidframework/ai-collab Package

Experimental package for utilities that enable/simplify interaction with LLMs for apps based on SharedTree.

See README.md for an overview of the package.

Interfaces

Interface	Alerts	Description
AiCollabErrorResponse	Alpha	An error response from the AI collaboration.
AiCollabOptions	Alpha	Options for the AI collaboration.
AiCollabSuccessResponse	Alpha	A successful response from the AI collaboration.
ApplyEditFailure	Alpha	An EventFlowDebugEvent marking the failure of applying an edit generated by the LLM to a SharedTree.
ApplyEditSuccess	Alpha	An EventFlowDebugEvent marking the successful application of an edit generated by the LLM to a SharedTree.
ArrayRangeRemoveDiff	Alpha	An object that describes the removal of a range of nodes from an array node.
ArraySingleRemoveDiff	Alpha	An object that describes the removal of a single node from an array node.
CoreEventLoopCompleted	Alpha	An EventFlowDebugEvent for signaling the end of the ai-collab's core event loop. There could be various reasons for the event loop to end, early exits and failures which should be captured in the status and failureReason fields. There will be exactly 1 of these events per ai-collab function execution.
CoreEventLoopStarted	Alpha	An EventFlowDebugEvent for signaling the start of the ai-collab's core event loop. Which makes various calls to the LLM to eventually apply edits to the users SharedTree which accomplish the user's provided goal. There will be exactly 1 of these events per ai-collab function execution.
DebugEvent	Alpha	Core Debug event type for the ai-collab
DiffBase	Alpha	A base interface to enforce consistency between all Diff objects.
DifferenceChange	Alpha	Represents a change operation between two branches of a tree. Meaning that an attribute (a shared tree node) was identified as being changed from one value to another.
DifferenceCreate	Alpha	Represents a create operation between two branches of a tree. Meaning that an attribute (a shared tree node) was identified as being created.
DifferenceMove	Alpha	Represents a move operation between two branches of a tree. Meaning that an object (shared tree node) was identified as being moved from one index to another based on its unique id.
DifferenceRemove	Alpha	Represents a remove operation between two branches of a tree. Meaning that an attribute (a shared tree node) was identified as being deleted. When using object ids, removes are idenitified by an object with a given id no longer existing.
EventFlowDebugEvent	Alpha	A Debug event that marks the start or end of a single core logic flow, such as generated tree edits, planning prompt, etc.
FinalReviewCompleted	Alpha	An EventFlowDebugEvent marking the end of the flow for prompting an LLM to complete a final review of its edits and determine whether the user's goal was accomplished.
FinalReviewStarted	Alpha	An EventFlowDebugEvent marking the initiation of the flow for prompting an LLM to complete a final review of its edits and determine whether the user's goal was accomplished.
GenerateTreeEditCompleted	Alpha	An EventFlowDebugEvent marking the completion of the flow for prompting an LLM to generate an LlmTreeEdit to the users Shared Tree based on the users initial ask. It is expected that the LLM will generate multiple of these events when it must generate multiple tree edits to satisfy the user request
GenerateTreeEditStarted	Alpha	An EventFlowDebugEvent marking the initiation of the flow for prompting an LLM to generate an LlmTreeEdit to the users Shared Tree based on the users initial ask. It is expected that the LLM will generate multiple of these events when it must generate multiple tree edits to satisfy the user request
InsertDiff	Alpha	An object that describes the insertion of a new node into a tree.
LlmApiCallDebugEvent	Alpha	An DebugEvent for an API call directly to a LLM.
ModifyDiff	Alpha	An object that describes the modification of an existing node on a tree.
MoveRangeDiff	Alpha	An object that describes the movement of a range of nodes from one array node to another array node.
MoveSingleDiff	Alpha	An object that describes the movement of a single node from one array node to another array node.
OpenAiClientOptions	Alpha	OpenAI client options for the AiCollabOptions interface.
Options	Alpha	Options for tree diffing.
PlanningPromptCompleted	Alpha	An EventFlowDebugEvent marking the completion of the flow for prompting an LLM to generate a plan for accomplishing the user's goal. There will be exactly 1 of these events per ai-collab function execution.
PlanningPromptStarted	Alpha	An EventFlowDebugEvent marking the initiation of the flow for prompting an LLM to generate a plan for accomplishing the user's goal. There will be exactly 1 of these events per ai-collab function execution.
RemoveNodeDiff	Alpha	Details about a node being removed from a field in an object node.
TokenLimits	Alpha	Maximum limits for the total tokens that can be used by an llm
TokenUsage	Alpha	Total usage of tokens by an LLM.

Classes

Class	Alerts	Description
SharedTreeBranchManager	Alpha	Manages determining the differences between two branches of a SharedTree represented as an actual tree node or a plain javascript object and applies said differences to the original SharedTree branch.

Types

TypeAlias	Alerts	Description
DebugEventLogHandler	Alpha	A callback function that can be used to handle debug events that occur during the AI collaboration process.
Diff	Alpha	An object that provides relevant information to visualize a single edit performed by an ai agent on a SharedTree
Difference	Alpha	Union for all possible difference types.
EventFlowDebugName	Alpha	The type for possible values for the eventFlowName field in an EventFlowDebugEvent.
LlmTreeEdit	Alpha	An edit generated by an LLM that can be applied to a given SharedTree.
MoveDiff	Alpha	An object that describes the movement of nodes from one array node to another array node.
NodePath	Alpha	A path from the root of the tree node passed to ai-collab to a specific node within the tree.
ObjectPath	Alpha	Represents a path through a tree of objects. number values represent array indices whereas string values represent object keys.
RemoveDiff	Alpha	An object that describes the removal of one or more nodes from a tree.

Functions

Function	Alerts	Return Type	Description
aiCollab(options)	Alpha	Promise<AiCollabSuccessResponse | AiCollabErrorResponse>	Calls an LLM to modify the provided SharedTree in a series of real time edits based on the provided users prompt input.
createMergableDiffSeries(diffs)	Alpha	Difference[]	Creates a set of mergeable diffs from a series of diffs produced by sharedTreeDiff(obj, newObj, options, _stack) that AREN'T using the object ID strategy. These diffs don't need any modifications to be applied to the old object.
createMergableIdDiffSeries(oldObject, diffs, idAttributeName)	Alpha	Difference[]	Creates a set of mergeable diffs from a series of diffs produced by sharedTreeDiff(obj, newObj, options, _stack) that are using the object ID strategy. These diffs don't need any modifications to be applied to the old object.
sharedTreeDiff(obj, newObj, options, _stack)	Alpha	Difference[]	Compares two objects and returns an array of differences between them.
sharedTreeTraverse(jsonObject, path)	Alpha	T | undefined	Traverses the provided ObjectPath on the provided Shared Tree or JSON object and returns the value at the end of the path.

Variables

Variable	Alerts	Modifiers	Type	Description
EventFlowDebugNames	Alpha	readonly	{ readonly CORE_EVENT_LOOP: "CORE_EVENT_LOOP"; readonly GENERATE_PLANNING_PROMPT: "GENERATE_PLANNING_PROMPT"; readonly GENERATE_AND_APPLY_TREE_EDIT: "GENERATE_AND_APPLY_TREE_EDIT"; readonly FINAL_REVIEW: "FINAL_REVIEW"; }	The possible values for the eventFlowName field in an EventFlowDebugEvent.

Function Details

aiCollab

Calls an LLM to modify the provided SharedTree in a series of real time edits based on the provided users prompt input.

This API is provided as an alpha preview and may change without notice.

To use, import via @fluidframework/ai-collab/alpha.

For more information about our API support guarantees, see here.

Signature

export declare function aiCollab(options: AiCollabOptions): Promise<AiCollabSuccessResponse | AiCollabErrorResponse>;

Remarks

Known Limitiations: - Root level array nodes are not supported - Nested arrays are not supported - Primitive nodes are not supported, e.g. 'string', 'number', 'boolean' - Your application's Shared Tree schema must have no more than 4 levels of nesting - Optional nodes are not supported in the Shared Tree schema - Union types are not supported in the Shared Tree schema - See README for more details.

Example

import {
	SchemaFactory,
	TreeViewConfiguration,
	type TreeView
} from "@fluidframework/tree";

const sf = new SchemaFactory("todo-app");

class TodoTask extends sf.object("TodoTask", {
	title: sf.string,
	description: sf.string,
}) {}

class TodoAppState extends sf.object("TodoAppState", {
	tasks: sf.array(TodoTask),
}) {}

// Initialize your SharedTree
const treeView: TreeView = tree.viewWith(new TreeViewConfiguration({ schema: TodoAppState }));
treeView.initialize({ tasks: [] });

// Collaborate with AI in realtime in just one function call.
const response = await aiCollab({
		openAI: {
			client: new OpenAI({
				apiKey: OPENAI_API_KEY,
			}),
			modelName: "gpt-4o",
		},
		treeNode: view.root,
		prompt: {
			systemRoleContext:
				"You are an helpful assistant managing a todo list for a user.",
			userAsk: "Create a set of new todos to plan a vacation to Cancun.",
		},
		planningStep: true,
		finalReviewStep: true,
	});

Parameters

Parameter	Type	Description
options	AiCollabOptions

Returns

Return type: Promise<AiCollabSuccessResponse | AiCollabErrorResponse>

createMergableDiffSeries

Creates a set of mergeable diffs from a series of diffs produced by sharedTreeDiff(obj, newObj, options, _stack) that AREN'T using the object ID strategy. These diffs don't need any modifications to be applied to the old object.

This API is provided as an alpha preview and may change without notice.

To use, import via @fluidframework/ai-collab/alpha.

For more information about our API support guarantees, see here.

Signature

export declare function createMergableDiffSeries(diffs: Difference[]): Difference[];

Parameters

Parameter	Type	Description
diffs	Difference[]

Returns

Return type: Difference[]

createMergableIdDiffSeries

Creates a set of mergeable diffs from a series of diffs produced by sharedTreeDiff(obj, newObj, options, _stack) that are using the object ID strategy. These diffs don't need any modifications to be applied to the old object.

This API is provided as an alpha preview and may change without notice.

To use, import via @fluidframework/ai-collab/alpha.

For more information about our API support guarantees, see here.

Signature

export declare function createMergableIdDiffSeries(oldObject: unknown, diffs: Difference[], idAttributeName: string | number): Difference[];

Parameters

Parameter	Type	Description
oldObject	unknown
diffs	Difference[]
idAttributeName	string | number

Returns

Return type: Difference[]

sharedTreeDiff

Compares two objects and returns an array of differences between them.

This API is provided as an alpha preview and may change without notice.

To use, import via @fluidframework/ai-collab/alpha.

For more information about our API support guarantees, see here.

Signature

export declare function sharedTreeDiff(obj: Record<string, unknown> | unknown[], newObj: Record<string, unknown> | unknown[], options?: Options, _stack?: (Record<string, unknown> | unknown[])[]): Difference[];

Parameters

Parameter	Modifiers	Type	Description
obj		Record<string, unknown> | unknown[]
newObj		Record<string, unknown> | unknown[]
options	optional	Options
_stack	optional	(Record<string, unknown> | unknown[])[]

Returns

Return type: Difference[]

sharedTreeTraverse

Traverses the provided ObjectPath on the provided Shared Tree or JSON object and returns the value at the end of the path.

This API is provided as an alpha preview and may change without notice.

To use, import via @fluidframework/ai-collab/alpha.

For more information about our API support guarantees, see here.

Signature

export declare function sharedTreeTraverse<T = unknown>(jsonObject: TreeMapNode | TreeArrayNode | Record<string, unknown>, path: ObjectPath): T | undefined;

Type Parameters

Parameter	Default	Description
T	unknown

Parameters

Parameter	Type	Description
jsonObject	TreeMapNode | TreeArrayNode | Record<string, unknown>
path	ObjectPath

Returns

Return type: T | undefined

Variable Details

EventFlowDebugNames

The possible values for the eventFlowName field in an EventFlowDebugEvent.

This API is provided as an alpha preview and may change without notice.

To use, import via @fluidframework/ai-collab/alpha.

For more information about our API support guarantees, see here.

Signature

EventFlowDebugNames: {
    readonly CORE_EVENT_LOOP: "CORE_EVENT_LOOP";
    readonly GENERATE_PLANNING_PROMPT: "GENERATE_PLANNING_PROMPT";
    readonly GENERATE_AND_APPLY_TREE_EDIT: "GENERATE_AND_APPLY_TREE_EDIT";
    readonly FINAL_REVIEW: "FINAL_REVIEW";
}

Type: { readonly CORE_EVENT_LOOP: "CORE_EVENT_LOOP"; readonly GENERATE_PLANNING_PROMPT: "GENERATE_PLANNING_PROMPT"; readonly GENERATE_AND_APPLY_TREE_EDIT: "GENERATE_AND_APPLY_TREE_EDIT"; readonly FINAL_REVIEW: "FINAL_REVIEW"; }