multimodality

A simple command line tool for text to image generation, using OpenAI's CLIP and a BigGAN. Technique was originally created by https://twitter.com/advadnoun

Automated modeling and machine learning framework FEDOT

An official implementation for "CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval"

A CLI tool/python module for generating images from text using guided diffusion and CLIP from OpenAI.

Sequence-to-Sequence Framework in PyTorch

A knowledge base construction engine for richly formatted data

X-VLM: Multi-Grained Vision Language Pre-Training (ICML 2022)

An official implementation for " UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation"

Attention-based multimodal fusion for sentiment analysis

DANCE: A Deep Learning Library and Benchmark Platform for Single-Cell Analysis

This repository contains code and metadata of How2 dataset

Pytorch implementation of CVPR2020 paper “VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation”

A Survey on multimodal learning research.

A deep learning framework for building multimodal multi-task learning systems.

A python framework accelerating ML based discovery in the medical field by encouraging code reuse. Batteries included :)

Code on selecting an action based on multimodal inputs. Here in this case inputs are voice and text.

A library of transformer models for computer vision and multi-modality research

中文领域的多模态Bert

multimodal social media content (text, image) classification

Fast regression and mediation analysis of vertex or voxel MRI data with TFCE