Scientific Research Publishing

Geo-Refined Point Transformer: Coordinate-Aware Excitation and Positional Upsampling for 3D Scene Segmentation ()

As a work exploring the existing trade-off between accuracy and efficiency in the context of point cloud processing, Point Transformer V3 (PTV3) has made significant advancements in computational ...
GitHub

Vision Transformer (ViT) for Image Classification

This project implements Vision Transformer (ViT) for image classification. Unlike CNNs, ViT splits images into patches and processes them as sequences using transformer architecture. It includes patch ...
GitHub

FEG: A New Geometric Positional Encoding for Long-Context Models

Instead of using RoPE’s low-dimensional limited rotations or ALiBi’s 1D linear bias, FEG builds position encoding on a higher-dimensional geometric structure. The idea is simple at a high level: Treat ...
IEEE

SPGFormer: Structure Perception Graph Transformer With Laplacian Position Encoding for Hyperspectral Image Classification

Abstract: With the integration of graph structure representation and self-attention mechanism, the graph Transformer (GT) demonstrates remarkable effectiveness in hyperspectral image (HSI) ...
Microsoft

Toward Relative Positional Encoding in Spiking Transformers

Spiking neural networks (SNNs) are bio-inspired networks that mimic how neurons in the brain communicate through discrete spikes, which have great potential in various tasks due to their energy ...