> For the complete documentation index, see [llms.txt](https://rudderlabs.gitbook.io/rudderlabs-1/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://rudderlabs.gitbook.io/rudderlabs-1/docs/transformations/index.md). # Transformations Transformations is one of RudderStack's key features. It gives you the ability to code custom JavaScript functions to implement specific use-cases on your event data, like: * Filtering/sampling events. * Enriching events by implementing static logic or leveraging an external API. * Cleaning/aggregating data. * Data masking or removing sensitive PII information in the events to ensure data privacy. * Implementing external actions on the events using an API. Transformations can be used across your [Event Stream](https://rudderstack.com/docs/stream-sources/), [Reverse ETL](https://rudderstack.com/docs/reverse-etl/) and [Cloud Extract](https://rudderstack.com/docs/cloud-extract-sources/) pipelines. You can refer to our [Sample Transformations](https://github.com/rudderlabs/sample-user-transformers) GitHub repository for some useful templates that you can use to create your own transformations. Transformations only work for the Cloud mode destinations. To know more about the Cloud mode in RudderStack, check out the [RudderStack Connection Modes](https://rudderstack.com/docs/rudderstack-cloud/rudderstack-connection-modes) guide. ## Adding a transformation * Log into your [**RudderStack dashboard**](https://app.rudderstack.com/). * Click on the [**Transformations**](https://app.rudderstack.com/transformations) link from the left panel in the dashboard, as shown: ![Adding a Transformation](/files/gnJFkGX1BNCprrogO8xf) * Click on the **Create New** option as shown: ![Create New Transformation](/files/XKMqVAf1SpLtDVhwR4y6) * Next, assign a name for this new transformation. Enter the transformation function's code in the **Transformation** window, as shown: ![Assign a transformation name](/files/AbNMxebUNuCVhcU2NEev) * Add your transformation code within the `transformEvent` function in the **Transformation** window. You can also add other functions and call them from within `transformEvent`. ![transformEvent function](/files/qI94FlEUbqELs259az9v) Our [Transformations](https://github.com/rudderlabs/sample-user-transformers) GitHub repository contains some useful templates that you can use to create your own transformations. You can copy-paste the entire code of any of the functions present in this repository into the **Transformation** window. Remember to delete the pre-populated `transformEvent` function in such cases, before pasting your code. * RudderStack also gives you the ability to test your transformation function with the **Run Test** option as shown: ![Run Test](/files/N9kldvCNcmMpybk4elTB) * In case you want to perform any aggregation/roll-up operation on a micro batch of events, use the `transformBatch` function, as shown: ```javascript export function transformBatch(events, metadata) { return events; } ``` Disclaimer for using `transformBatch`: * Make sure you pass on the `messageId` from the input event onto the output event. If the `messageId` is passed on, the order of delivery of events is preserved. **Without it, this order may not be maintained**. * It is highly recommended to use `transformEvent` in every possible case as it ensures event ordering in all cases. * In case a `transformBatch` is required and event ordering is important, make sure to pass on the `messageId`. ## Libraries One of the key features of RudderStack Transformations is the ability to reuse code written for a transformation in other transformations as well. For this, RudderStack lets you create libraries or functions that can be reused in different transformations. To create a library, follow these steps: * Click on the [**Transformations**](https://app.rudderstack.com/transformations) link in the left nav bar and go to the **Libraries** section. Click on the **Create New** option to add new libraries, as shown: ![Create new library](/files/nPItpcl0QSsEVFNDEXXb) * Add the library's **Name**, an optional **Description**, and include the custom functions that you need to reuse across all other transformations. ![Add library name and description](/files/s1rREOVovARY1m5qAGAr) * You can add multiple functions to a single library, as shown: ![Add more functions under a single library](/files/IxKkIsCwPjyMJVBYijgT) RudderStack does not support the deletion of libraries as of now. This is to ensure that you don't break any existing transformations that use the library that you might be trying to delete. ## Using libraries in transformations To use the libraries in your existing transformation, simply take the name of your library and convert it into **camel case without spaces** - this becomes your library handle. For example, if your library name is **`is rudder email`**, then the library handle would be **`isRudderEmail`**. ![Using libraries in transformations](/files/ruHyIy75mUYZNGaIN376) Let's say you want to import a function called **`rudderEmail`**, which returns `true` for the emails from a specific domain (e.g. RudderStack) and `false` otherwise, from the **`is rudder email`** library. Also, you want to use this function to filter the events that don't have the email address of the specified domain. The following code snippet demonstrates how we can implement this use-case: ```javascript import { rudderEmail } from "isRudderEmail"; export function transformEvent(event) { const email = event.context && event.context.traits && event.context.traits.email; if (email) { if (!rudderEmail(email)) return; } return event; } ``` On running a test, an example event not having the specified email domain is filtered out, as shown: ![Event filtered out without specifying email domain](/files/TiSd4TPeBocGa9Wi3Oie) ### Importing multiple functions from a single library When importing a single function or multiple functions from a library, it is important to keep the import statement to one line. Breaking the import function into multiple lines will lead to an error. The following snippets highlight how to properly import functions from a library: ```javascript // Correct Way // --------------- import { getLoss } from "getFinanceData"; // OR import { getLoss, getRevenue, getProfit } from "getFinanceData"; import { getLoss, getRevenue, getProfit } from "getFinanceData"; // For default getPrice import import getPrice, { getRevenue, getProfit } from "getFinanceData"; // alias imports import getPrice as gp, { getRevenue as gr, getProfit } from "getFinanceData"; // usuage: gp(event), gr(event), getProfit(ev) import * as GFD from "getFinanceData"; // usuage: GFD.getRevenue(ev), GFD.getProfit(ev) // for default import: GFD.default(ev) ``` ```javascript // Incorrect Way // ----------------- import * from "getFinanceData"; // OR import getPrice as gp from "getFinanceData"; getPrice(ev) ``` ## Accessing metadata RudderStack injects a function `metadata(event)` into your transformations as an argument. This allows you to access the event metadata variables that help you customize your transformations. `metadata()` takes the event as input and returns the metadata of the event. The following properties, if available, are present in the metadata response: | Property Name | Description | | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `sourceId` | This refers to the ID of the source configured on your RudderStack dashboard. Note that it different from the source **Write Key**. Refer to the image below for more details. | | `destinationId` | ID of the destination configured on your RudderStack dashboard. | | `messageId` | Corresponds to the unique ID for each event. | | `sessionId` | If sessions are enabled, this corresponds to the value of the session ID. | ![Source ID](/files/cLbH5EggFTnJkoi9pa7i) Source ID ![Source ID](/files/JASytUgZKAyLEsGJhOwO) Since you may not need the metadata in every transformation, you can optionally access it by including it in your function signature as an argument wherever required. An example of this is as shown below: ```javascript export function transformEvent(event, metadata) { const meta = metadata(event); event.sourceId = meta.sourceId; return event; } ``` ## External API requests You can make any number of external API requests in your transformation functions and use the response to enrich your events data. RudderStack injects an asynchronous `fetch(url)` function into your transformations. It makes an API call to the given URL and returns the response in a JSON format. An example of how to use the `fetch` function in transformations is shown below: ```javascript export async function transformEvent(event, metadata) { const res = await fetch("post_url", { method: "POST", // POST, PUT, DELETE, GET, etc. headers: { "Content-Type": "application/json;charset=UTF-8", Authorization: "Bearer " }, body: JSON.stringify(event) }); event.response = JSON.stringify(res); return event; } ``` To see how this feature is used in more detail, check out the [Clearbit enrichment example](https://github.com/rudderlabs/sample-user-transformers/blob/sampleTransformationV1/EnrichWithClearbit.js) which uses the `fetch` function. For improved performance, we recommend using `batch` API requests instead of a separate API request for each event whenever possible. ### FetchV2 FetchV2 is a wrapper around the `fetch` call to provide more access to fetch the response properties. The following properties are present in a `fetchV2` response: | Property Name | Description | | ------------- | ------------------------------------------------------ | | `status` | Status code of fetch response, e.g. `200`. | | `url` | URL of the Fetch API. | | `headers` | Response headers | | `body` | Response body in JSON or TEXT. By default, it is JSON. | Use the `try-catch` block to capture the cause for failures during the `fetch` call, e.g. when a timeout is applied. An example of how to use the `fetchV2` function in RudderStack transformations is shown below: ```javascript export async function transformEvent(event) { try { const res = await fetchV2("url", { timeout: 1000}); if (res.status == 200) { event.response = JSON.stringify(res.body); } } catch (err) { log(err.message); } return event; } ``` ## Debugging with logs You can access event-related logs while running a test by including the `log()` function in your transformation code. An example of this is as shown: ```javascript export function transformEvent(event, metadata) { const meta = metadata(event); event.sourceId = meta.sourceId; log("Event Name is", event.event, ";", "Message Id is", event.messageId); log("Source Id is", meta.sourceId); return event; } ``` On running a test on the above code, you can see the logs in the **Logs** section of the dashboard, as shown: ![Adding a Transformation](/files/4dOc7eOLJHRwOEwBivfB) Arguments to the `log` function can be a String, Number or Object. ## Limits You should take into account the memory and time limits when adding a new transformation. Each invocation of the transformation should not exceed the following limits: | Parameter | Limit | | ------------ | --------- | | Memory Limit | 8 MB | | Time Limit | 4 seconds | The user transformation fails if these limits are exceeded. ## Contact us For queries on any of the sections covered in this guide, you can [**contact us**](mailto:%20docs@rudderstack.com) or start a conversation in our [**Slack**](https://rudderstack.com/join-rudderstack-slack-community) community. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://rudderlabs.gitbook.io/rudderlabs-1/docs/transformations/index.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.